You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Hong Tang (JIRA)" <ji...@apache.org> on 2010/06/02 09:11:37 UTC
[jira] Created: (MAPREDUCE-1833) [gridmix3] limit the maximum task
duration in sleep job.
[gridmix3] limit the maximum task duration in sleep job.
--------------------------------------------------------
Key: MAPREDUCE-1833
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1833
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: contrib/gridmix
Reporter: Hong Tang
In production job history logs, sometimes a task takes very long time to finish. Replaying such trace in sleep-job mode in Gridmix3 would unnecessarily prolong the benchmark execution time. It would be desirable to allow users to limit the maximum task duration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Need Suggestion: Tuning MR performance by changing parameters in Hadoop project and JVM
Posted by WANG Shicai <Ev...@yahoo.cn>.
Hi,
This message is a little long. So I beg your patient.
Our team would like to tune MR performance by changing parameters in Hadoop project and JVM according to the MR Job status and result.
First, classify MR jobs into several kinds. Then monitor cpu, memory, etc. in a MR job, structing the data from the monitor and input it into HBase. The crucial step is to build a model or models to analysis the data. Finally, acquire the proposal for tuning MR jobs, such as increase the memory for the job or reduce it, etc.
However, I am a developer in HBase subproject and not so acquainted with MR jobs. I need some suggestion about the following aspects:
* Is this plan feasible or not? why?
* Is there any one or team doing the above before?
* Which processes in a MR job we ought to monitor more carefully?
* Which parameters in that processes we ought to care?
* What can we refer for the model building?
* Also, any other suggestion about our plan will be welcome.
Thank you a lot!!!
Evan,
2010-06-02
__________________________________________________
�Ͽ�ע���Ż����������������?
http://cn.mail.yahoo.com
[jira] Resolved: (MAPREDUCE-1833) [gridmix3] limit the maximum task
duration in sleep job.
Posted by "Hong Tang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang resolved MAPREDUCE-1833.
----------------------------------
Resolution: Duplicate
The changes will be incorporated in MAPREDUCE-1936.
> [gridmix3] limit the maximum task duration in sleep job.
> --------------------------------------------------------
>
> Key: MAPREDUCE-1833
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1833
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/gridmix
> Reporter: Hong Tang
> Attachments: mr-1833-yahoo-20.10.patch, mr-1833-yahoo-20.1xx.patch
>
>
> In production job history logs, sometimes a task takes very long time to finish. Replaying such trace in sleep-job mode in Gridmix3 would unnecessarily prolong the benchmark execution time. It would be desirable to allow users to limit the maximum task duration.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.