You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Vinay Kumar Thota (Created) (JIRA)" <ji...@apache.org> on 2012/02/07 13:33:04 UTC
[jira] [Created] (MAPREDUCE-3831) RM scalability runtime is worse
than 0.20.204 by 14.2%
RM scalability runtime is worse than 0.20.204 by 14.2%
------------------------------------------------------
Key: MAPREDUCE-3831
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3831
Project: Hadoop Map/Reduce
Issue Type: Task
Components: benchmarks, resourcemanager
Affects Versions: 0.23.1
Reporter: Vinay Kumar Thota
RM scalability runtime is worse than 0.20.204 by 14.2%
Overall runtime against Fred in a 350 nodes cluster is 2155 secs.
Overall runtime against .23 in 350 nodes cluster is 2462 secs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3831) RM scalability runtime is worse
than 0.20.204 by 14.2%
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202656#comment-13202656 ]
Arun C Murthy commented on MAPREDUCE-3831:
------------------------------------------
This looks like a bug in GridMix. Amar, can you please take a look? Thanks.
> RM scalability runtime is worse than 0.20.204 by 14.2%
> ------------------------------------------------------
>
> Key: MAPREDUCE-3831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3831
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: benchmarks, mrv2, resourcemanager
> Affects Versions: 0.23.1
> Reporter: Vinay Kumar Thota
>
> RM scalability runtime is worse than 0.20.204 by 14.2%
> Overall runtime against Fred in a 350 nodes cluster is 2155 secs.
> Overall runtime against .23 in 350 nodes cluster is 2462 secs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3831) RM scalability runtime is worse
than 0.20.204 by 14.2%
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3831:
-----------------------------------------------
Issue Type: Sub-task (was: Task)
Parent: MAPREDUCE-3561
> RM scalability runtime is worse than 0.20.204 by 14.2%
> ------------------------------------------------------
>
> Key: MAPREDUCE-3831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3831
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: benchmarks, mrv2, resourcemanager
> Affects Versions: 0.23.1
> Reporter: Vinay Kumar Thota
>
> RM scalability runtime is worse than 0.20.204 by 14.2%
> Overall runtime against 0.20.204 in a 350 nodes cluster is 2155 secs.
> Overall runtime against .23 in 350 nodes cluster is 2462 secs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3831) RM scalability runtime is worse
than 0.20.204 by 14.2%
Posted by "Vinod Kumar Vavilapalli (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli resolved MAPREDUCE-3831.
------------------------------------------------
Resolution: Fixed
Thanks for the update Amol.
Closing this as fixed, couple of the tickets tracked under MAPREDUCE-3561 automatically addressed this I suppose.
> RM scalability runtime is worse than 0.20.204 by 14.2%
> ------------------------------------------------------
>
> Key: MAPREDUCE-3831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3831
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: benchmarks, mrv2, resourcemanager
> Affects Versions: 0.23.1
> Reporter: Vinay Kumar Thota
>
> RM scalability runtime is worse than 0.20.204 by 14.2%
> Overall runtime against 0.20.204 in a 350 nodes cluster is 2155 secs.
> Overall runtime against .23 in 350 nodes cluster is 2462 secs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3831) RM scalability runtime is worse
than 0.20.204 by 14.2%
Posted by "Mahadev konar (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-3831:
-------------------------------------
Component/s: mrv2
> RM scalability runtime is worse than 0.20.204 by 14.2%
> ------------------------------------------------------
>
> Key: MAPREDUCE-3831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3831
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: benchmarks, mrv2, resourcemanager
> Affects Versions: 0.23.1
> Reporter: Vinay Kumar Thota
>
> RM scalability runtime is worse than 0.20.204 by 14.2%
> Overall runtime against Fred in a 350 nodes cluster is 2155 secs.
> Overall runtime against .23 in 350 nodes cluster is 2462 secs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3831) RM scalability runtime is worse
than 0.20.204 by 14.2%
Posted by "Vinay Kumar Thota (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinay Kumar Thota updated MAPREDUCE-3831:
-----------------------------------------
Description:
RM scalability runtime is worse than 0.20.204 by 14.2%
Overall runtime against 0.20.204 in a 350 nodes cluster is 2155 secs.
Overall runtime against .23 in 350 nodes cluster is 2462 secs.
was:
RM scalability runtime is worse than 0.20.204 by 14.2%
Overall runtime against Fred in a 350 nodes cluster is 2155 secs.
Overall runtime against .23 in 350 nodes cluster is 2462 secs.
> RM scalability runtime is worse than 0.20.204 by 14.2%
> ------------------------------------------------------
>
> Key: MAPREDUCE-3831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3831
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: benchmarks, mrv2, resourcemanager
> Affects Versions: 0.23.1
> Reporter: Vinay Kumar Thota
>
> RM scalability runtime is worse than 0.20.204 by 14.2%
> Overall runtime against 0.20.204 in a 350 nodes cluster is 2155 secs.
> Overall runtime against .23 in 350 nodes cluster is 2462 secs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3831) RM scalability runtime is worse
than 0.20.204 by 14.2%
Posted by "Karam Singh (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202431#comment-13202431 ]
Karam Singh commented on MAPREDUCE-3831:
----------------------------------------
Settings for For RM scalability comparison :-
GridMix settings were for both Hadoop-0.20.204 and Hadoop-0.23 :
{code}
gridmix.client.submit.threads=10
gridmix.submit.multiplier=0.01
gridmix.client.pending.queue.depth=100
gridmix.job.type=SLEEPJOB
gridmix.sleep.maptask-only=false
gridmix.sleep.max-map-time=300000
gridmix.sleep.max-reduce-tim=300000
gridmix.sleep.fake-locations=3
gridmix.distributed-cache-emulation.enable=falseemulation.enable>
gridmix.compression-emulation.enable=false
{code}
>From JobHistroy Parsing and GridMix client found that :
Runtime (seconds): 2473
GridMix Simulation Time Spent: 41mins 8sec
Workflow End: 2046 (From histroy parsing)
While looking at GridMix log and JobHistory files :
1. Found that according to gm client last was completed was
{code}
12/02/07 08:32:26 INFO gridmix.JobMonitor: GRIDMIX000029 job_1328600848949_1182) success.
Whereas the
EndTime of Job is : 1328602818684 Tue, 07 Feb 2012 08:20:18
{code}
Which means somehow GridMix got Job completion event 12 minutes after the the actual job got completed .
2. Similarly acc. to JobHistory last Jo completed was :
{code}
job_1328600848949_1162: 1328603121882 Tue, 07 Feb 2012 08:25:21
Whereas according to GridMix client log:
12/02/07 08:32:08 INFO gridmix.JobMonitor: GRIDMIX000029 (job_1328600848949_1162) success
{code}
Which again means GridMix got job completion event nearly 7 minutes after the actual job got finished.
Whereas this problem does not exists with Hadoop-0.20.204
Seems that, Somehow in Hadoop-0.23, GridMix is getting job completion events long after the actual job getting completed
> RM scalability runtime is worse than 0.20.204 by 14.2%
> ------------------------------------------------------
>
> Key: MAPREDUCE-3831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3831
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: benchmarks, resourcemanager
> Affects Versions: 0.23.1
> Reporter: Vinay Kumar Thota
>
> RM scalability runtime is worse than 0.20.204 by 14.2%
> Overall runtime against Fred in a 350 nodes cluster is 2155 secs.
> Overall runtime against .23 in 350 nodes cluster is 2462 secs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3831) RM scalability runtime is worse
than 0.20.204 by 14.2%
Posted by "Amol Kekre (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209433#comment-13209433 ]
Amol Kekre commented on MAPREDUCE-3831:
---------------------------------------
pls close this jira. RM scalability now matches .204
> RM scalability runtime is worse than 0.20.204 by 14.2%
> ------------------------------------------------------
>
> Key: MAPREDUCE-3831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3831
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: benchmarks, mrv2, resourcemanager
> Affects Versions: 0.23.1
> Reporter: Vinay Kumar Thota
>
> RM scalability runtime is worse than 0.20.204 by 14.2%
> Overall runtime against 0.20.204 in a 350 nodes cluster is 2155 secs.
> Overall runtime against .23 in 350 nodes cluster is 2462 secs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira