You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Vinay Kumar Thota (JIRA)" <ji...@apache.org> on 2010/11/29 11:36:37 UTC
[jira] Created: (MAPREDUCE-2204) Implement gridmix system tests
with different time intervals of high ram job traces.
Implement gridmix system tests with different time intervals of high ram job traces.
-------------------------------------------------------------------------------------
Key: MAPREDUCE-2204
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2204
Project: Hadoop Map/Reduce
Issue Type: Task
Components: contrib/gridmix
Reporter: Vinay Kumar Thota
Assignee: Vinay Kumar Thota
Implement gridmix system tests with different time intervals of High Ram map reduce jobs with below scenarios.
1) Generate input data based on cluster size and create the synthetic jobs by using the 2 min MR High RAM jobs trace and submit the jobs with below arguments.
GRIDMIX_JOB_TYPE = SleepJob
GRIDMIX_USER_RESOLVER = SubmitterUserResolver
GRIDMIX_SUBMISSION_POLICY = SERIAL
GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = false
Input Size = 200 MB * No. of nodes in cluster.
TRACE_FILE = 5 min folded trace.
GRIDMIX_SLEEP_MAP_MAX_TIME=5 sec.
GRIDMIX_SLEEP_REDUCE_MAX_TIME=5 sec.
Verify JobStatus for each job and summary (QueueName, UserName, StatTime, FinishTime, MAPS, REDUCERS and COUNTERS etc) after completion of execution.
2) Generate input data based on cluster size and create the synthetic jobs by using the 3 min MR High RAM jobs trace and submit the jobs with below arguments.
GRIDMIX_JOB_TYPE = LoadJob
GRIDMIX_USER_RESOLVER = RoundRobinUserResolver
GRIDMIX_SUBMISSION_POLICY = STRESS
GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = false
BYTES_PER_FILE = 200 MB
Input Size = 400 MB * No. of nodes in cluster.
TRACE_FILE = 3 min folded trace.
Verify JobStatus for each job and summary (QueueName, UserName, StatTime, FinishTime, MAPS, REDUCERS and COUNTERS etc) after completion of execution.
3) Generate input data based on cluster size and create the synthetic jobs by using the 5 min MR High RAM jobs trace and submit the jobs with below arguments.
GRIDMIX_JOB_TYPE = LoadJob
GRIDMIX_USER_RESOLVER = EchoUserResolver
GRIDMIX_SUBMISSION_POLICY = Replay
GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = false
Input Size = 300 MB * No. of nodes in cluster.
TRACE_FILE = 5 min folded trace.
Verify JobStatus for each job and summary (QueueName, UserName, StatTime, FinishTime, MAPS, REDUCERS and COUNTERS etc) after completion of execution.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Resolved] (MAPREDUCE-2204) Implement gridmix system tests
with different time intervals of high ram job traces.
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amar Kamat resolved MAPREDUCE-2204.
-----------------------------------
Tags: gridmix system-tests
Resolution: Duplicate
Fix Version/s: 0.23.0
Committed as part of MAPREDUCE-2517.
> Implement gridmix system tests with different time intervals of high ram job traces.
> -------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2204
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2204
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: contrib/gridmix
> Reporter: Vinay Kumar Thota
> Assignee: Vinay Kumar Thota
> Fix For: 0.23.0
>
>
> Implement gridmix system tests with different time intervals of High Ram map reduce jobs with below scenarios.
> 1) Generate input data based on cluster size and create the synthetic jobs by using the 2 min MR High RAM jobs trace and submit the jobs with below arguments.
> GRIDMIX_JOB_TYPE = SleepJob
> GRIDMIX_USER_RESOLVER = SubmitterUserResolver
> GRIDMIX_SUBMISSION_POLICY = SERIAL
> GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = false
> Input Size = 200 MB * No. of nodes in cluster.
> TRACE_FILE = 5 min folded trace.
> GRIDMIX_SLEEP_MAP_MAX_TIME=5 sec.
> GRIDMIX_SLEEP_REDUCE_MAX_TIME=5 sec.
> Verify JobStatus for each job and summary (QueueName, UserName, StatTime, FinishTime, MAPS, REDUCERS and COUNTERS etc) after completion of execution.
> 2) Generate input data based on cluster size and create the synthetic jobs by using the 3 min MR High RAM jobs trace and submit the jobs with below arguments.
> GRIDMIX_JOB_TYPE = LoadJob
> GRIDMIX_USER_RESOLVER = RoundRobinUserResolver
> GRIDMIX_SUBMISSION_POLICY = STRESS
> GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = false
> BYTES_PER_FILE = 200 MB
> Input Size = 400 MB * No. of nodes in cluster.
> TRACE_FILE = 3 min folded trace.
> Verify JobStatus for each job and summary (QueueName, UserName, StatTime, FinishTime, MAPS, REDUCERS and COUNTERS etc) after completion of execution.
> 3) Generate input data based on cluster size and create the synthetic jobs by using the 5 min MR High RAM jobs trace and submit the jobs with below arguments.
> GRIDMIX_JOB_TYPE = LoadJob
> GRIDMIX_USER_RESOLVER = EchoUserResolver
> GRIDMIX_SUBMISSION_POLICY = Replay
> GRIDMIX_JOB_SUBMISSION_QUEUE_IN_TRACE = false
> Input Size = 300 MB * No. of nodes in cluster.
> TRACE_FILE = 5 min folded trace.
> Verify JobStatus for each job and summary (QueueName, UserName, StatTime, FinishTime, MAPS, REDUCERS and COUNTERS etc) after completion of execution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira