You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Devaraj Das (JIRA)" <ji...@apache.org> on 2007/03/09 07:59:24 UTC
[jira] Created: (HADOOP-1101) Add more statistics in the web-ui to
do with tasks
Add more statistics in the web-ui to do with tasks
--------------------------------------------------
Key: HADOOP-1101
URL: https://issues.apache.org/jira/browse/HADOOP-1101
Project: Hadoop
Issue Type: Improvement
Reporter: Devaraj Das
Assigned To: Devaraj Das
Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. The rate of reading map input and writing reduce output will be helpful too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1101) Add more statistics in the web-ui to
do with tasks
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-1101:
--------------------------------
Fix Version/s: 0.12.1
Description: Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. (was: Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. The rate of reading map input and writing reduce output will be helpful too.)
Affects Version/s: 0.12.0
> Add more statistics in the web-ui to do with tasks
> --------------------------------------------------
>
> Key: HADOOP-1101
> URL: https://issues.apache.org/jira/browse/HADOOP-1101
> Project: Hadoop
> Issue Type: Improvement
> Affects Versions: 0.12.0
> Reporter: Devaraj Das
> Assigned To: Devaraj Das
> Fix For: 0.12.1
>
> Attachments: 1101.patch
>
>
> Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1101) Add more statistics in the web-ui to
do with tasks
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1101:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Devaraj.
> Add more statistics in the web-ui to do with tasks
> --------------------------------------------------
>
> Key: HADOOP-1101
> URL: https://issues.apache.org/jira/browse/HADOOP-1101
> Project: Hadoop
> Issue Type: Improvement
> Affects Versions: 0.12.0
> Reporter: Devaraj Das
> Assigned To: Devaraj Das
> Fix For: 0.13.0
>
> Attachments: 1101.patch
>
>
> Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1101) Add more statistics in the web-ui to
do with tasks
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-1101:
--------------------------------
Attachment: 1101.patch
This patch does the following (in analysejobhistory.jsp):
1) Splits the Reduce phase into two - shuffle and the actual "reduce". It displays the data for each seperately.
2) Displays the average time, and the transition time for the last task for each of the phases (map, shuffle, reduce).
> Add more statistics in the web-ui to do with tasks
> --------------------------------------------------
>
> Key: HADOOP-1101
> URL: https://issues.apache.org/jira/browse/HADOOP-1101
> Project: Hadoop
> Issue Type: Improvement
> Reporter: Devaraj Das
> Assigned To: Devaraj Das
> Attachments: 1101.patch
>
>
> Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. The rate of reading map input and writing reduce output will be helpful too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1101) Add more statistics in the web-ui to
do with tasks
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-1101:
--------------------------------
Fix Version/s: (was: 0.12.1)
0.13.0
deferring to 0.13.0
> Add more statistics in the web-ui to do with tasks
> --------------------------------------------------
>
> Key: HADOOP-1101
> URL: https://issues.apache.org/jira/browse/HADOOP-1101
> Project: Hadoop
> Issue Type: Improvement
> Affects Versions: 0.12.0
> Reporter: Devaraj Das
> Assigned To: Devaraj Das
> Fix For: 0.13.0
>
> Attachments: 1101.patch
>
>
> Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1101) Add more statistics in the web-ui to
do with tasks
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-1101:
--------------------------------
Status: Patch Available (was: Open)
> Add more statistics in the web-ui to do with tasks
> --------------------------------------------------
>
> Key: HADOOP-1101
> URL: https://issues.apache.org/jira/browse/HADOOP-1101
> Project: Hadoop
> Issue Type: Improvement
> Reporter: Devaraj Das
> Assigned To: Devaraj Das
> Attachments: 1101.patch
>
>
> Currently, the web-ui does not display all the statistics needed to analyze performance of a job in the MR framework. Things like the average time the maps took to finish is missing. Similarly, the min/max/average for shuffle is missing. It would also be nice to add the statistics for major transitions like the time when the last map finished, the time when the last shuffle/copy finished, etc. The rate of reading map input and writing reduce output will be helpful too.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.