You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "vitthal (Suhas) Gogate (Created) (JIRA)" <ji...@apache.org> on 2011/10/18 20:23:10 UTC
[jira] [Created] (MAPREDUCE-3202) Integrating Hadoop Vaidya with
JobHistory Server
Integrating Hadoop Vaidya with JobHistory Server
------------------------------------------------
Key: MAPREDUCE-3202
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3202
Project: Hadoop Map/Reduce
Issue Type: New Feature
Components: jobhistoryserver
Affects Versions: 0.20.205.1
Reporter: vitthal (Suhas) Gogate
At present jobdetailshistory page served by JobHistory Server provides elementary job analysis through link "Analyze This job". Hadoop Vaidya provides a detailed analysis of the M/R job in terms of various execution inefficiencies and the associated remedies that user can easily understand and fix. Integrating Hadoop Vaidya with JobHistory server would really improve the usability of this tool and also benefit many naive users understanding various performance problems and/or best practices violations associated with their job.
Integration would also aim at providing users a convenient interface where they can manage the existing rules as well as write their own new rules.
During my tenure at Yahoo, Vaidya tool was successfully deployed in production analyzing tens of thousands of jobs every day with lot more useful rules than the sample ones present in the contrib project. Many of these rules are open sourced already (big thanks to Yahoo! MAPREDUCE-1530) but yet to integrate with the tool.
I will add more design details for this feature in near future as work towards getting prototype running.. Any thoughts/comments are welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3202) Integrating Hadoop Vaidya with
JobHistory Server
Posted by "vitthal (Suhas) Gogate (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
vitthal (Suhas) Gogate updated MAPREDUCE-3202:
----------------------------------------------
Affects Version/s: 0.20.205.0
> Integrating Hadoop Vaidya with JobHistory Server
> ------------------------------------------------
>
> Key: MAPREDUCE-3202
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3202
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobhistoryserver
> Affects Versions: 0.20.205.0, 1.0.0
> Reporter: vitthal (Suhas) Gogate
>
> At present jobdetailshistory page served by JobHistory Server provides elementary job analysis through link "Analyze This job". Hadoop Vaidya provides a detailed analysis of the M/R job in terms of various execution inefficiencies and the associated remedies that user can easily understand and fix. Integrating Hadoop Vaidya with JobHistory server would really improve the usability of this tool and also benefit many naive users understanding various performance problems and/or best practices violations associated with their job.
> Integration would also aim at providing users a convenient interface where they can manage the existing rules as well as write their own new rules.
> During my tenure at Yahoo, Vaidya tool was successfully deployed in production analyzing tens of thousands of jobs every day with lot more useful rules than the sample ones present in the contrib project. Many of these rules are open sourced already (big thanks to Yahoo! MAPREDUCE-1530) but yet to integrate with the tool.
> I will add more design details for this feature in near future as work towards getting prototype running.. Any thoughts/comments are welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira