You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Amar Kamat (JIRA)" <ji...@apache.org> on 2011/06/14 13:55:47 UTC
[jira] [Created] (MAPREDUCE-2591) [Gridmix] Improve cumulative CPU
usage emulation
[Gridmix] Improve cumulative CPU usage emulation
------------------------------------------------
Key: MAPREDUCE-2591
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2591
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: contrib/gridmix
Reporter: Amar Kamat
Assignee: Amar Kamat
Fix For: 0.23.0
MAPREDUCE-2106 introduced a basic cumulative cpu usage emulation in Gridmix. There are some known cases where it can be improved. The cases are
1. JVM reuse
2. Short lived maps emulating high CPU usage
3. Long running maps/reducers with high CPU usage
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2591) [Gridmix] Improve cumulative
CPU usage emulation
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049186#comment-13049186 ]
Amar Kamat commented on MAPREDUCE-2591:
---------------------------------------
Adding details:
1. JVM reuse: Here the total cpu usage of the process tree will be a net sum of all the other tasks that ran in the current JVM. This information is not known to the task. Cpu usage emulator should identify such cases and emulate on top of the current cpu usage.
2. Short lived maps emulating high CPU usage: If the total span of the map task is short, then the task will exit before emulating the resource usage. The framework (or the emulator) should detect such situations (based on map input data size, input records, features like compression etc) and emulate aggressively. On way to do this would be to give more weight in the early phases of the task.
3. Long running maps/reducers with high CPU usage: If the total runtime of the task is high and the total expected cpu usage is high, then the emulator should speed up its emulation phase. One option can be to use multi-threading. Other option would be to switch to a more cpu intensive operation(s).
> [Gridmix] Improve cumulative CPU usage emulation
> ------------------------------------------------
>
> Key: MAPREDUCE-2591
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2591
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/gridmix
> Reporter: Amar Kamat
> Assignee: Amar Kamat
> Fix For: 0.23.0
>
>
> MAPREDUCE-2106 introduced a basic cumulative cpu usage emulation in Gridmix. There are some known cases where it can be improved. The cases are
> 1. JVM reuse
> 2. Short lived maps emulating high CPU usage
> 3. Long running maps/reducers with high CPU usage
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira