You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Karam Singh (Created) (JIRA)" <ji...@apache.org> on 2011/12/09 12:04:40 UTC
[jira] [Created] (MAPREDUCE-3524) Scan runtime is more than 1.5x
slower in 0.23
Scan runtime is more than 1.5x slower in 0.23
---------------------------------------------
Key: MAPREDUCE-3524
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Karam Singh
Priority: Critical
Scan runtime is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3524) Scan benchmark is more than 1.5x
slower in 0.23
Posted by "Vinod Kumar Vavilapalli (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli resolved MAPREDUCE-3524.
------------------------------------------------
Resolution: Fixed
> Scan benchmark is more than 1.5x slower in 0.23
> -----------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: benchmarks, mr-am, mrv2, performance
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Priority: Blocker
>
> Scan benchmark is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3524) Scan benchmark is more than
1.5x slower in 0.23
Posted by "Amol Kekre (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213064#comment-13213064 ]
Amol Kekre commented on MAPREDUCE-3524:
---------------------------------------
pls close the jira
> Scan benchmark is more than 1.5x slower in 0.23
> -----------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: benchmarks, mr-am, mrv2, performance
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
>
> Scan benchmark is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3524) Scan runtime is more than 1.5x
slower in 0.23
Posted by "Aaron T. Myers (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166422#comment-13166422 ]
Aaron T. Myers commented on MAPREDUCE-3524:
-------------------------------------------
Hey Karam, can you explain exactly what you mean by "scan runtime?" Thanks a lot.
> Scan runtime is more than 1.5x slower in 0.23
> ---------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Priority: Critical
>
> Scan runtime is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3524) Scan runtime is more than 1.5x
slower in 0.23
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3524:
-----------------------------------------------
Issue Type: Sub-task (was: Bug)
Parent: MAPREDUCE-3561
> Scan runtime is more than 1.5x slower in 0.23
> ---------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
>
> Scan runtime is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3524) Scan runtime is more than 1.5x
slower in 0.23
Posted by "Arun C Murthy (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy reassigned MAPREDUCE-3524:
----------------------------------------
Assignee: Vinod Kumar Vavilapalli
> Scan runtime is more than 1.5x slower in 0.23
> ---------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
>
> Scan runtime is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3524) Scan benchmark is more than 1.5x
slower in 0.23
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3524:
-----------------------------------------------
Assignee: (was: Vinod Kumar Vavilapalli)
Thanks for the update Amol.
This was one of the crazier benchmarks where we had to do lot of work.
Unassigning it from myself - this was a team effort, thanks to Sid, Arun and Karam also!
> Scan benchmark is more than 1.5x slower in 0.23
> -----------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: benchmarks, mr-am, mrv2, performance
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Priority: Blocker
>
> Scan benchmark is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3524) Scan benchmark is more than
1.5x slower in 0.23
Posted by "Amol Kekre (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195194#comment-13195194 ]
Amol Kekre commented on MAPREDUCE-3524:
---------------------------------------
Performance has improved from 92% slower to 38% slower. Still not there yet...
> Scan benchmark is more than 1.5x slower in 0.23
> -----------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: benchmarks, mr-am, mrv2, performance
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
>
> Scan benchmark is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3524) Scan runtime is more than 1.5x
slower in 0.23
Posted by "Amol Kekre (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amol Kekre updated MAPREDUCE-3524:
----------------------------------
Priority: Blocker (was: Critical)
> Scan runtime is more than 1.5x slower in 0.23
> ---------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Priority: Blocker
>
> Scan runtime is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3524) Scan benchmark is more than 1.5x
slower in 0.23
Posted by "Vinod Kumar Vavilapalli (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod Kumar Vavilapalli updated MAPREDUCE-3524:
-----------------------------------------------
Component/s: performance
mr-am
benchmarks
Description:
Scan benchmark is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
was:
Scan runtime is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
Summary: Scan benchmark is more than 1.5x slower in 0.23 (was: Scan runtime is more than 1.5x slower in 0.23)
I should've edited the title before itself. Scan is one of the mapreduce benchmarks. It is same as the main loaden benchmark with specific settings. (See GenericMRLoadGenerator.java). Scan is to measure the job runtime for simply generating random data and reading+emitting the key-values (only maps, no reduces).
> Scan benchmark is more than 1.5x slower in 0.23
> -----------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: benchmarks, mr-am, mrv2, performance
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
>
> Scan benchmark is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3524) Scan benchmark is more than
1.5x slower in 0.23
Posted by "Amol Kekre (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213063#comment-13213063 ]
Amol Kekre commented on MAPREDUCE-3524:
---------------------------------------
Performance is almost on par:
Runtime: 2.36% worse. Scan throughput: 1.9% worse.
> Scan benchmark is more than 1.5x slower in 0.23
> -----------------------------------------------
>
> Key: MAPREDUCE-3524
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3524
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: benchmarks, mr-am, mrv2, performance
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Vinod Kumar Vavilapalli
> Priority: Blocker
>
> Scan benchmark is more than 1.5X slower(almost 92% increased) in 0.23 than Hadoop-0.20.204 on 350 nodes size cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira