You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2009/10/15 21:19:31 UTC
[jira] Created: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Capacity Scheduler scheduling information is hard to read / should be tabular format
------------------------------------------------------------------------------------
Key: MAPREDUCE-1118
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Allen Wittenauer
The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880733#action_12880733 ]
Hadoop QA commented on MAPREDUCE-1118:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12447576/mapred-1118.patch
against trunk revision 956335.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 3 new Findbugs warnings.
-1 release audit. The applied patch generated 2 release audit warnings (more than the trunk's current 0 warnings).
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/256/testReport/
Release audit warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/256/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/256/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/256/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/256/console
This message is automatically generated.
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated MAPREDUCE-1118:
----------------------------------------
Affects Version/s: 0.20.2
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Attachment: mapred-1118-2.patch
minor revision from previous version
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118-2.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Status: Open (was: Patch Available)
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118-2.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Status: Patch Available (was: Open)
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Attachment: mapred-1118-3.patch
Incorporate review recommendations (Dick)
Have taken care of the "serious ones"
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118-2.patch, mapred-1118-3.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran reassigned MAPREDUCE-1118:
-----------------------------------------------
Assignee: Krishna Ramachandran
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Attachment: mapred-1118.20S.patch
Fix for an earlier release not for commit in trunk
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Attachment: mapred-1118-1.patch
patch revised
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Status: Patch Available (was: Open)
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118-2.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766468#action_12766468 ]
Allen Wittenauer commented on MAPREDUCE-1118:
---------------------------------------------
Flipping between multiple pages to present the same info is even worse. It basically means I'd need a web browser window per queue. :(
So yes, make this a new ask then: a decent, readable summary of all queue information such that one could intelligently compare what is happening in each queue in relation to the other queues. See also the Fair Share Scheduler's /scheduler page as an example.
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Allen Wittenauer
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898174#action_12898174 ]
Hemanth Yamijala commented on MAPREDUCE-1118:
---------------------------------------------
I took this patch for a spin. Without going into too much detail of the code, I could see some high level points to discuss:
- The /scheduler page that is being added does not have all the fields for the queues - it only has fields related to capacity parameters. The queueinfo page on the other hand has more fields - like tasks, limits, job counts etc . I would imagine we'll need more information in the servlet, no ? Allen ?
- The patch doesn't play well with hierarchical queues introduced in Hadoop 0.21 (MAPREDUCE-853). When I configured a simple hierarchy: parent level queues: p1 and p2, each having one child queue: q1 and q2 respectively, I got the following exception when I accessed /scheduler:
{code}
java.lang.NullPointerException
at org.apache.hadoop.mapred.CapacitySchedulerServlet.showQueues(CapacitySchedulerServlet.java:127)
at org.apache.hadoop.mapred.CapacitySchedulerServlet.doGet(CapacitySchedulerServlet.java:90)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
{code}
More importantly, I am not even sure *what* should be displayed in the case of hierarchical queues that makes sense to meet Allen's original requirement - i.e. provide an easy-to-read interface to compare queue information. The confusion clearly is that queues across hierarchies do not make sense to be compared (unless the information is normalized at some global level). So, probably what makes sense is to have this kind of tabular structure for queues at every level. Clearly, this issue does not arise for Hadoop 0.20.
- From an interface point of view, I think we can do better in how scheduling information is accessed from the main page. This information has been available via the 'queues' link, and this patch adds another entry point - the /scheduler page. Perhaps discussion around the hierarchical queue interface will give us ideas around this as well.
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118-2.patch, mapred-1118-3.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Status: Patch Available (was: Open)
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118-2.patch, mapred-1118-3.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874936#action_12874936 ]
Hemanth Yamijala commented on MAPREDUCE-1118:
---------------------------------------------
Would the same apply for Command line as well ? mapred queue -info ?
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Status: Patch Available (was: Open)
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766463#action_12766463 ]
Vinod K V commented on MAPREDUCE-1118:
--------------------------------------
On trunk, the queue information is moved into a new page queueinfo.jsp which is linked to from jobtracker.jsp. One can now look at information for each queue separately, at any time information of only one queue is presented. That should address the readability problem.
If we want cross queue-comparison, that is a new ask.
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Allen Wittenauer
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Attachment: mapred-1118.patch
Added a servlet to tabulate CapacityTaskScheduler queue stats
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Status: Open (was: Patch Available)
patch revised
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12875249#action_12875249 ]
Allen Wittenauer commented on MAPREDUCE-1118:
---------------------------------------------
The command line needs machine parse-able output that is scheduler agnostic. But I sort of view that as a separate issue.
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Dick King (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897532#action_12897532 ]
Dick King commented on MAPREDUCE-1118:
--------------------------------------
This comment is a review.
First, let me say that I didn't review {{sorttable.js}} . It would be bad to have subtly different versions of this code flying around.
{{CapacitySchedulerServlet.java}}
near end of {{doGet()}} :
*This is serious*: {{ByteArrayOutputSteam.writeTo(OutputStream)}} throws. Please revise this call to something like
{noformat}
OutputStream servletOut = null;
try {
servletOut = response.getOutputStream();
baos.writeTo(servletOut);
} finally {
if (servletOut != null) {
servletOut.close();
}
}
{noformat}
.
*This is semi-serious*: In {{showQueues}} , where queues are printed, the code
{noformat}
out.printf(
"<td><a href=\"jobqueue_details.jsp?queueName=%s\">%s</a></td>\n",
name, name);
{noformat}
the code deposits the name right in the middle of hard-core HTML. If the queue names contain obnoxious characters such as a quote or an angle bracket we could have a bad day. These characters should be escaped with HTML escape sequences such as {{<}} , etc.
Don't forget to escape the ampersands :-) . I believe that only quote marks and angle brackets need to be escaped in the URL, but everything needs to be escaped in the rendered text.
*This is a nit*: In
{noformat}
out.printf("<td>%s</td>\n", queuesManager.getJobQueue(name)
.getRunningJobs().size());
out.printf("<td>%s</td>\n", qsc.getNumOfWaitingJobs());
{noformat}
I can't condone dropping numeric data onto a {{%s}} . I realize that it works but it looks ugly to my eye.
*This is potentially serious*: I don't see where {{showQueues}} does the needed locking. You allude to this by defensively dumping into a {{ByteArrayOutputStream}} , but the code doesn't lock anything. I can see why it should. Can queues disappear or appear?
*This is a potential omission*: The block comment before the {{class}} declaration claims to implement an advanced mode, but I don't see any footprint of such a thing in the code. In any event, I'm not a big fan of magic URLs. The servlet should include a button to bring itself into advanced mode. If there are users that shouldn't be able to go into advanced mode, this should be handled in some other manner than hidden URLs.
I don't see the code to get into the scheduler manager servlet. Perhaps there should be a button in the job tracker administration page when the capacity scheduler is in use?
{{TaskSchedulingMgr}}
{{infoServer.setAttribute("scheduler", this);}}
*This is a nit*: I would prefer {{infoServer.setAttribute("scheduler.scheduler", this);}} . All of the servlets share an attribute namespace. However, this one isn't bad as such things go, since it's hard to imagine another servlet code author putting anything except the ambient scheduler into that attribute.
{{TestCapacitySchedulerServlet}}
This is a minor nit.
I can't condone {{assertTrue(queueData.contains("50.0%"));}} . That's the moral equivalent of floating point equality. I do realize that 1/2 can be represented exactly in most float systems, but you might want to do something else, even if only allowing the value to be {{49.9}} which is okay because the servlet does print it out as a {{%.1f}} .
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118-2.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1118) Capacity Scheduler scheduling
information is hard to read / should be tabular format
Posted by "Krishna Ramachandran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krishna Ramachandran updated MAPREDUCE-1118:
--------------------------------------------
Status: Open (was: Patch Available)
> Capacity Scheduler scheduling information is hard to read / should be tabular format
> ------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1118
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1118
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 0.20.2
> Reporter: Allen Wittenauer
> Assignee: Krishna Ramachandran
> Attachments: mapred-1118-1.patch, mapred-1118-2.patch, mapred-1118.20S.patch, mapred-1118.patch
>
>
> The scheduling information provided by the capacity scheduler is extremely hard to read on the job tracker web page. Instead of just flat text, it should be presenting the information in a tabular format, similar to what the fair share scheduler provides. This makes it much easier to compare what different queues are doing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.