You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "patrick white (Created) (JIRA)" <ji...@apache.org> on 2012/02/27 18:04:49 UTC
[jira] [Created] (MAPREDUCE-3928) App GUI Cluster->Applications UI
has confusing job status report
App GUI Cluster->Applications UI has confusing job status report
----------------------------------------------------------------
Key: MAPREDUCE-3928
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3928
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: applicationmaster
Affects Versions: 0.23.1
Reporter: patrick white
The 0.23.1 Application GUI has some potential usability issues and confusion
points with respect to job status, from a user's perspective.
Currently, when starting from the App UI base link of
http://<RM_HOST>:8088/cluster, the main window shows "All Applications", and in
the upper left corner is the "Cluster" pulldown, opened up showing the
following list:
About
Nodes
Applications
- New
- Submitted
- Accepted
- Running
- Finished
- Failed
- Killed
Scheduler
When jobs are submitted, they show up in the pre-final states, like Accepted
and Running. however after completion, no matter on the final outcome of a job
(SUCCESS, KILLED or FAILED), all jobs are listed under FINISHED, this is okay
since any completion result qualifies as Finished, but the FAILED and KILLED
jobs do not appear in the corresponding links for FAILED or KILLED.
Clicking on FAILED or KILLED reports "No data available in table", even with
Failed or Killed jobs showing up in the FINISHED link. This can be very
confusing to a user checking on their jobs, especially for someone using the
direct URL links, such as:
http://<RM_HOST>:8088/cluster/apps/FAILED
http://<RM_HOST>:8088/cluster/apps/KILLED
Another potential issue is that the RM and AM each have their own
interpretation of a job's result, so the State and FinalStatus reported in the
Cluster Metrics display may not align with the defined states in the Cluster
pulldown. It would be useful to clearly delineate areas of the GUI wrt the
component visible states of a user's job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3928) App GUI Cluster->Applications
UI has confusing job status report
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217638#comment-13217638 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3928:
----------------------------------------------------
bq. Another potential issue is that the RM and AM each have their own interpretation of a job's result, so the State and FinalStatus reported in the Cluster Metrics display may not align with the defined states in the Cluster pulldown.
This is the main point. Let me clarify:
- The final state of the application is fundamentally different from the FinalStatus of the application.
- FinalStatus of an application is a separate API for custom frameworks to expose their final status.
- On the RM UI, the state always represents the state of the application.
- In MapReduce case, a Failed/Killed job still corresponds to a FINISHED application.
Given that, is it that you are expecting Failed/Killed mapreduce jobs to appear under FAILED/KILLED applications list?
> App GUI Cluster->Applications UI has confusing job status report
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-3928
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3928
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 0.23.1
> Reporter: patrick white
>
> The 0.23.1 Application GUI has some potential usability issues and confusion
> points with respect to job status, from a user's perspective.
> Currently, when starting from the App UI base link of
> http://<RM_HOST>:8088/cluster, the main window shows "All Applications", and in
> the upper left corner is the "Cluster" pulldown, opened up showing the
> following list:
> About
> Nodes
> Applications
> - New
> - Submitted
> - Accepted
> - Running
> - Finished
> - Failed
> - Killed
> Scheduler
> When jobs are submitted, they show up in the pre-final states, like Accepted
> and Running. however after completion, no matter on the final outcome of a job
> (SUCCESS, KILLED or FAILED), all jobs are listed under FINISHED, this is okay
> since any completion result qualifies as Finished, but the FAILED and KILLED
> jobs do not appear in the corresponding links for FAILED or KILLED.
> Clicking on FAILED or KILLED reports "No data available in table", even with
> Failed or Killed jobs showing up in the FINISHED link. This can be very
> confusing to a user checking on their jobs, especially for someone using the
> direct URL links, such as:
> http://<RM_HOST>:8088/cluster/apps/FAILED
> http://<RM_HOST>:8088/cluster/apps/KILLED
> Another potential issue is that the RM and AM each have their own
> interpretation of a job's result, so the State and FinalStatus reported in the
> Cluster Metrics display may not align with the defined states in the Cluster
> pulldown. It would be useful to clearly delineate areas of the GUI wrt the
> component visible states of a user's job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3928) App GUI Cluster->Applications
UI has confusing job status report
Posted by "Anupam Seth (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252008#comment-13252008 ]
Anupam Seth commented on MAPREDUCE-3928:
----------------------------------------
Agree that it is an issue of user education for now and hence closing the JIRA.
In the future, though, we could potentially look at some interesting possibilities to reduce the learning curve for users transitioning from the 1.0 / 0.20 world such as having an MR specific GUI or maybe easier fixes such as column based filtering on the RM, but that is lower priority and can be dealt with as the community deems the best approach with an enhancement request at that time.
> App GUI Cluster->Applications UI has confusing job status report
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-3928
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3928
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 0.23.1
> Reporter: patrick white
> Assignee: Anupam Seth
>
> The 0.23.1 Application GUI has some potential usability issues and confusion
> points with respect to job status, from a user's perspective.
> Currently, when starting from the App UI base link of
> http://<RM_HOST>:8088/cluster, the main window shows "All Applications", and in
> the upper left corner is the "Cluster" pulldown, opened up showing the
> following list:
> About
> Nodes
> Applications
> - New
> - Submitted
> - Accepted
> - Running
> - Finished
> - Failed
> - Killed
> Scheduler
> When jobs are submitted, they show up in the pre-final states, like Accepted
> and Running. however after completion, no matter on the final outcome of a job
> (SUCCESS, KILLED or FAILED), all jobs are listed under FINISHED, this is okay
> since any completion result qualifies as Finished, but the FAILED and KILLED
> jobs do not appear in the corresponding links for FAILED or KILLED.
> Clicking on FAILED or KILLED reports "No data available in table", even with
> Failed or Killed jobs showing up in the FINISHED link. This can be very
> confusing to a user checking on their jobs, especially for someone using the
> direct URL links, such as:
> http://<RM_HOST>:8088/cluster/apps/FAILED
> http://<RM_HOST>:8088/cluster/apps/KILLED
> Another potential issue is that the RM and AM each have their own
> interpretation of a job's result, so the State and FinalStatus reported in the
> Cluster Metrics display may not align with the defined states in the Cluster
> pulldown. It would be useful to clearly delineate areas of the GUI wrt the
> component visible states of a user's job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3928) App GUI Cluster->Applications
UI has confusing job status report
Posted by "Jason Lowe (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13227856#comment-13227856 ]
Jason Lowe commented on MAPREDUCE-3928:
---------------------------------------
Yes, I think the confusion here is that a user is primarily interested in whether their job succeeded, failed, was killed, etc. The filter links on the apps page implies one can quickly filter the apps to one of those states, yet apps that have failed or killed often don't show up under the FAILED or KILLED filters. It's not clear on the web page that those only filter State instead of the FinalState, and many users may not understand or appreciate the distinction between State and FinalState (at least for MapReduce apps).
For example, it can be a bit confusing why a job that's killed via mapred job -kill shows up as KILLED / KILLED, while another job where the AM process is killed directly is listed as FINISHED / KILLED.
> App GUI Cluster->Applications UI has confusing job status report
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-3928
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3928
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 0.23.1
> Reporter: patrick white
>
> The 0.23.1 Application GUI has some potential usability issues and confusion
> points with respect to job status, from a user's perspective.
> Currently, when starting from the App UI base link of
> http://<RM_HOST>:8088/cluster, the main window shows "All Applications", and in
> the upper left corner is the "Cluster" pulldown, opened up showing the
> following list:
> About
> Nodes
> Applications
> - New
> - Submitted
> - Accepted
> - Running
> - Finished
> - Failed
> - Killed
> Scheduler
> When jobs are submitted, they show up in the pre-final states, like Accepted
> and Running. however after completion, no matter on the final outcome of a job
> (SUCCESS, KILLED or FAILED), all jobs are listed under FINISHED, this is okay
> since any completion result qualifies as Finished, but the FAILED and KILLED
> jobs do not appear in the corresponding links for FAILED or KILLED.
> Clicking on FAILED or KILLED reports "No data available in table", even with
> Failed or Killed jobs showing up in the FINISHED link. This can be very
> confusing to a user checking on their jobs, especially for someone using the
> direct URL links, such as:
> http://<RM_HOST>:8088/cluster/apps/FAILED
> http://<RM_HOST>:8088/cluster/apps/KILLED
> Another potential issue is that the RM and AM each have their own
> interpretation of a job's result, so the State and FinalStatus reported in the
> Cluster Metrics display may not align with the defined states in the Cluster
> pulldown. It would be useful to clearly delineate areas of the GUI wrt the
> component visible states of a user's job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3928) App GUI Cluster->Applications UI
has confusing job status report
Posted by "Anupam Seth (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth resolved MAPREDUCE-3928.
------------------------------------
Resolution: Won't Fix
> App GUI Cluster->Applications UI has confusing job status report
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-3928
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3928
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 0.23.1
> Reporter: patrick white
> Assignee: Anupam Seth
>
> The 0.23.1 Application GUI has some potential usability issues and confusion
> points with respect to job status, from a user's perspective.
> Currently, when starting from the App UI base link of
> http://<RM_HOST>:8088/cluster, the main window shows "All Applications", and in
> the upper left corner is the "Cluster" pulldown, opened up showing the
> following list:
> About
> Nodes
> Applications
> - New
> - Submitted
> - Accepted
> - Running
> - Finished
> - Failed
> - Killed
> Scheduler
> When jobs are submitted, they show up in the pre-final states, like Accepted
> and Running. however after completion, no matter on the final outcome of a job
> (SUCCESS, KILLED or FAILED), all jobs are listed under FINISHED, this is okay
> since any completion result qualifies as Finished, but the FAILED and KILLED
> jobs do not appear in the corresponding links for FAILED or KILLED.
> Clicking on FAILED or KILLED reports "No data available in table", even with
> Failed or Killed jobs showing up in the FINISHED link. This can be very
> confusing to a user checking on their jobs, especially for someone using the
> direct URL links, such as:
> http://<RM_HOST>:8088/cluster/apps/FAILED
> http://<RM_HOST>:8088/cluster/apps/KILLED
> Another potential issue is that the RM and AM each have their own
> interpretation of a job's result, so the State and FinalStatus reported in the
> Cluster Metrics display may not align with the defined states in the Cluster
> pulldown. It would be useful to clearly delineate areas of the GUI wrt the
> component visible states of a user's job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3928) App GUI Cluster->Applications UI
has confusing job status report
Posted by "Anupam Seth (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anupam Seth reassigned MAPREDUCE-3928:
--------------------------------------
Assignee: Anupam Seth
> App GUI Cluster->Applications UI has confusing job status report
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-3928
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3928
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 0.23.1
> Reporter: patrick white
> Assignee: Anupam Seth
>
> The 0.23.1 Application GUI has some potential usability issues and confusion
> points with respect to job status, from a user's perspective.
> Currently, when starting from the App UI base link of
> http://<RM_HOST>:8088/cluster, the main window shows "All Applications", and in
> the upper left corner is the "Cluster" pulldown, opened up showing the
> following list:
> About
> Nodes
> Applications
> - New
> - Submitted
> - Accepted
> - Running
> - Finished
> - Failed
> - Killed
> Scheduler
> When jobs are submitted, they show up in the pre-final states, like Accepted
> and Running. however after completion, no matter on the final outcome of a job
> (SUCCESS, KILLED or FAILED), all jobs are listed under FINISHED, this is okay
> since any completion result qualifies as Finished, but the FAILED and KILLED
> jobs do not appear in the corresponding links for FAILED or KILLED.
> Clicking on FAILED or KILLED reports "No data available in table", even with
> Failed or Killed jobs showing up in the FINISHED link. This can be very
> confusing to a user checking on their jobs, especially for someone using the
> direct URL links, such as:
> http://<RM_HOST>:8088/cluster/apps/FAILED
> http://<RM_HOST>:8088/cluster/apps/KILLED
> Another potential issue is that the RM and AM each have their own
> interpretation of a job's result, so the State and FinalStatus reported in the
> Cluster Metrics display may not align with the defined states in the Cluster
> pulldown. It would be useful to clearly delineate areas of the GUI wrt the
> component visible states of a user's job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3928) App GUI Cluster->Applications
UI has confusing job status report
Posted by "Vinod Kumar Vavilapalli (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250924#comment-13250924 ]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3928:
----------------------------------------------------
bq. yet apps that have failed or killed often don't show up under the FAILED or KILLED filters
I think you mean "jobs that have failed or killed"..
bq. it can be a bit confusing why a job that's killed via mapred job -kill shows up as KILLED / KILLED
I am planning to change this. "mapred job -kill" should only affect job-state. We need to have a "app -kill" which works at an application level. I already have a WIP patch from some time back which we should get in.
bq. many users may not understand or appreciate the distinction between State and FinalState (at least for MapReduce apps).
This is the fundamental problem we should address. It is happening because the concept of an application is new for mapreduce users.
IMO, we need to educate the users the distinction between applications vs jobs and appliation-state vs job-state.
If we agree to that, we can close this down as invalid and work on improving documentation to make the distinction between application and job very explicit.
> App GUI Cluster->Applications UI has confusing job status report
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-3928
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3928
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
> Affects Versions: 0.23.1
> Reporter: patrick white
> Assignee: Anupam Seth
>
> The 0.23.1 Application GUI has some potential usability issues and confusion
> points with respect to job status, from a user's perspective.
> Currently, when starting from the App UI base link of
> http://<RM_HOST>:8088/cluster, the main window shows "All Applications", and in
> the upper left corner is the "Cluster" pulldown, opened up showing the
> following list:
> About
> Nodes
> Applications
> - New
> - Submitted
> - Accepted
> - Running
> - Finished
> - Failed
> - Killed
> Scheduler
> When jobs are submitted, they show up in the pre-final states, like Accepted
> and Running. however after completion, no matter on the final outcome of a job
> (SUCCESS, KILLED or FAILED), all jobs are listed under FINISHED, this is okay
> since any completion result qualifies as Finished, but the FAILED and KILLED
> jobs do not appear in the corresponding links for FAILED or KILLED.
> Clicking on FAILED or KILLED reports "No data available in table", even with
> Failed or Killed jobs showing up in the FINISHED link. This can be very
> confusing to a user checking on their jobs, especially for someone using the
> direct URL links, such as:
> http://<RM_HOST>:8088/cluster/apps/FAILED
> http://<RM_HOST>:8088/cluster/apps/KILLED
> Another potential issue is that the RM and AM each have their own
> interpretation of a job's result, so the State and FinalStatus reported in the
> Cluster Metrics display may not align with the defined states in the Cluster
> pulldown. It would be useful to clearly delineate areas of the GUI wrt the
> component visible states of a user's job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira