You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinay Kumar Thota (JIRA)" <ji...@apache.org> on 2010/07/23 09:44:50 UTC
[jira] Created: (MAPREDUCE-1963) [Herriot] TaskMemoryManager should
log process-tree's status while killing tasks
[Herriot] TaskMemoryManager should log process-tree's status while killing tasks
--------------------------------------------------------------------------------
Key: MAPREDUCE-1963
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1963
Project: Hadoop Map/Reduce
Issue Type: Task
Components: test
Reporter: Vinay Kumar Thota
1. Execute a streaming job which will increase memory usage beyond configured memory limits during mapping phase. TaskMemoryManager should logs a map task's process-tree's status just before killing the task.
2. Execute a streaming job which will increase memory usage beyond configured memory limits during reduce phase. TaskMemoryManager should logs a reduce task's process-tree's status just before killing the task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1963) [Herriot] TaskMemoryManager
should log process-tree's status while killing tasks
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893047#action_12893047 ]
Konstantin Boudnik commented on MAPREDUCE-1963:
-----------------------------------------------
Please have patch for the trunk as well.
> [Herriot] TaskMemoryManager should log process-tree's status while killing tasks
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1963
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1963
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1963-ydist-security.patch
>
>
> 1. Execute a streaming job which will increase memory usage beyond configured memory limits during mapping phase. TaskMemoryManager should logs a map task's process-tree's status just before killing the task.
> 2. Execute a streaming job which will increase memory usage beyond configured memory limits during reduce phase. TaskMemoryManager should logs a reduce task's process-tree's status just before killing the task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1963) [Herriot] TaskMemoryManager
should log process-tree's status while killing tasks
Posted by "Balaji Rajagopalan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892188#action_12892188 ]
Balaji Rajagopalan commented on MAPREDUCE-1963:
-----------------------------------------------
Actual test case states that you should grep for "PID PPID PGRPID SESSID CMD_NAME VMEM_USAGE(BYTES) FULL_CMD_LINE". I dont see this happening in the current test cases everything else looks good.
> [Herriot] TaskMemoryManager should log process-tree's status while killing tasks
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1963
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1963
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1963-ydist-security.patch
>
>
> 1. Execute a streaming job which will increase memory usage beyond configured memory limits during mapping phase. TaskMemoryManager should logs a map task's process-tree's status just before killing the task.
> 2. Execute a streaming job which will increase memory usage beyond configured memory limits during reduce phase. TaskMemoryManager should logs a reduce task's process-tree's status just before killing the task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1963) [Herriot] TaskMemoryManager
should log process-tree's status while killing tasks
Posted by "Balaji Rajagopalan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892231#action_12892231 ]
Balaji Rajagopalan commented on MAPREDUCE-1963:
-----------------------------------------------
I was'nt asking you hard code regular expression, my comment was I did not find for cmd_name in your regular expression for example, you explained to me cmd name will be part of *, full_cmd_line will be also grepped as * I guess from your regular expression. It is not clear looking at the regular expression what exactly the data will be greped, if you can add an example as the comment that will be helpful.
> [Herriot] TaskMemoryManager should log process-tree's status while killing tasks
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1963
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1963
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1963-ydist-security.patch
>
>
> 1. Execute a streaming job which will increase memory usage beyond configured memory limits during mapping phase. TaskMemoryManager should logs a map task's process-tree's status just before killing the task.
> 2. Execute a streaming job which will increase memory usage beyond configured memory limits during reduce phase. TaskMemoryManager should logs a reduce task's process-tree's status just before killing the task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1963) [Herriot] TaskMemoryManager should
log process-tree's status while killing tasks
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-1963:
-----------------------------------------
Assignee: Vinay Kumar Thota
> [Herriot] TaskMemoryManager should log process-tree's status while killing tasks
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1963
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1963
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
>
> 1. Execute a streaming job which will increase memory usage beyond configured memory limits during mapping phase. TaskMemoryManager should logs a map task's process-tree's status just before killing the task.
> 2. Execute a streaming job which will increase memory usage beyond configured memory limits during reduce phase. TaskMemoryManager should logs a reduce task's process-tree's status just before killing the task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1963) [Herriot] TaskMemoryManager should
log process-tree's status while killing tasks
Posted by "Konstantin Boudnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Boudnik updated MAPREDUCE-1963:
------------------------------------------
Tags: herriot
> [Herriot] TaskMemoryManager should log process-tree's status while killing tasks
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1963
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1963
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1963-ydist-security.patch
>
>
> 1. Execute a streaming job which will increase memory usage beyond configured memory limits during mapping phase. TaskMemoryManager should logs a map task's process-tree's status just before killing the task.
> 2. Execute a streaming job which will increase memory usage beyond configured memory limits during reduce phase. TaskMemoryManager should logs a reduce task's process-tree's status just before killing the task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1963) [Herriot] TaskMemoryManager
should log process-tree's status while killing tasks
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892226#action_12892226 ]
Vinay Kumar Thota commented on MAPREDUCE-1963:
----------------------------------------------
Verifying the task diagnostics through regular expression and we can't hardcode the dump in the test because the dump will vary from run to run. If you see the test,there is regular expression string(taskOverLimitPatternString).
> [Herriot] TaskMemoryManager should log process-tree's status while killing tasks
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1963
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1963
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1963-ydist-security.patch
>
>
> 1. Execute a streaming job which will increase memory usage beyond configured memory limits during mapping phase. TaskMemoryManager should logs a map task's process-tree's status just before killing the task.
> 2. Execute a streaming job which will increase memory usage beyond configured memory limits during reduce phase. TaskMemoryManager should logs a reduce task's process-tree's status just before killing the task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1963) [Herriot] TaskMemoryManager
should log process-tree's status while killing tasks
Posted by "Balaji Rajagopalan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892232#action_12892232 ]
Balaji Rajagopalan commented on MAPREDUCE-1963:
-----------------------------------------------
+1 from me since the above is just adding a comment.
> [Herriot] TaskMemoryManager should log process-tree's status while killing tasks
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1963
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1963
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1963-ydist-security.patch
>
>
> 1. Execute a streaming job which will increase memory usage beyond configured memory limits during mapping phase. TaskMemoryManager should logs a map task's process-tree's status just before killing the task.
> 2. Execute a streaming job which will increase memory usage beyond configured memory limits during reduce phase. TaskMemoryManager should logs a reduce task's process-tree's status just before killing the task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1963) [Herriot] TaskMemoryManager should
log process-tree's status while killing tasks
Posted by "Vinay Kumar Thota (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-1963:
-----------------------------------------
Attachment: 1963-ydist-security.patch
patch for yahoo security dist branch.
> [Herriot] TaskMemoryManager should log process-tree's status while killing tasks
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1963
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1963
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: test
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Attachments: 1963-ydist-security.patch
>
>
> 1. Execute a streaming job which will increase memory usage beyond configured memory limits during mapping phase. TaskMemoryManager should logs a map task's process-tree's status just before killing the task.
> 2. Execute a streaming job which will increase memory usage beyond configured memory limits during reduce phase. TaskMemoryManager should logs a reduce task's process-tree's status just before killing the task.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.