You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ravi Prakash (JIRA)" <ji...@apache.org> on 2012/10/05 22:30:02 UTC
[jira] [Created] (MAPREDUCE-4711) Append time elapsed since
job-start-time for finished tasks
Ravi Prakash created MAPREDUCE-4711:
---------------------------------------
Summary: Append time elapsed since job-start-time for finished tasks
Key: MAPREDUCE-4711
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4711
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobhistoryserver
Affects Versions: 0.23.3
Reporter: Ravi Prakash
Assignee: Ravi Prakash
In 0.20.x/1.x, the analyze job link gave this information
bq. The last Map task task_<sometask> finished at (relative to the Job launch time): 5/10 20:23:10 (1hrs, 27mins, 54sec)
The time it took for the last task to finish needs to be calculated mentally in 0.23. I believe we should print it next to the finish time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4711) Append time elapsed since
job-start-time for finished tasks
Posted by "Ravi Prakash (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Prakash updated MAPREDUCE-4711:
------------------------------------
Attachment: MAPREDUCE-4711.branch-0.23.patch
This patch adds that information to the webpage
> Append time elapsed since job-start-time for finished tasks
> -----------------------------------------------------------
>
> Key: MAPREDUCE-4711
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4711
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
> Affects Versions: 0.23.3
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
> Attachments: MAPREDUCE-4711.branch-0.23.patch
>
>
> In 0.20.x/1.x, the analyze job link gave this information
> bq. The last Map task task_<sometask> finished at (relative to the Job launch time): 5/10 20:23:10 (1hrs, 27mins, 54sec)
> The time it took for the last task to finish needs to be calculated mentally in 0.23. I believe we should print it next to the finish time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4711) Append time elapsed since
job-start-time for finished tasks
Posted by "Ravi Prakash (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Prakash updated MAPREDUCE-4711:
------------------------------------
Status: Patch Available (was: Open)
> Append time elapsed since job-start-time for finished tasks
> -----------------------------------------------------------
>
> Key: MAPREDUCE-4711
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4711
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
> Affects Versions: 0.23.3
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
> Attachments: MAPREDUCE-4711.branch-0.23.patch
>
>
> In 0.20.x/1.x, the analyze job link gave this information
> bq. The last Map task task_<sometask> finished at (relative to the Job launch time): 5/10 20:23:10 (1hrs, 27mins, 54sec)
> The time it took for the last task to finish needs to be calculated mentally in 0.23. I believe we should print it next to the finish time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4711) Append time elapsed since
job-start-time for finished tasks
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471696#comment-13471696 ]
Robert Joseph Evans commented on MAPREDUCE-4711:
------------------------------------------------
The patch looks OK to me, but I don't know if I like seeing all of the extra text in the table. It starts to make the table very crowded and busy and it already is too overcrowded for my taste. I would much rather have something visual then have a huge table with all of this information in it, but that is probably a ways off. Perhaps we could look at adding in something to show/hide columns so we don't need to see everything, and if someone wants to dig deeper into an issue they can add in more columns. Alternatively I would not mind having an analyze this job link like in 1.0, at least until something better can be determined.
I also noticed that there appears to be a bug in the history server. It is reporting the task start time to be the same as the job start time (or at least the time that the task was initialized by the AM). In both 1.x and in the AM the task start time is the start time of the first task attempt. We need to file a JIRA for this too or else your new time is always going to be the same as the column right next to it.
> Append time elapsed since job-start-time for finished tasks
> -----------------------------------------------------------
>
> Key: MAPREDUCE-4711
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4711
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
> Affects Versions: 0.23.3
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
> Attachments: MAPREDUCE-4711.branch-0.23.patch
>
>
> In 0.20.x/1.x, the analyze job link gave this information
> bq. The last Map task task_<sometask> finished at (relative to the Job launch time): 5/10 20:23:10 (1hrs, 27mins, 54sec)
> The time it took for the last task to finish needs to be calculated mentally in 0.23. I believe we should print it next to the finish time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4711) Append time elapsed since
job-start-time for finished tasks
Posted by "Ravi Prakash (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472490#comment-13472490 ]
Ravi Prakash commented on MAPREDUCE-4711:
-----------------------------------------
Bobby! I agree with you. The table already was, and I've made it more, crowded. I am going to be working on adding more intuitive visual representations of the job progress. Graphs and all. My vision is that the table will only serve as a repository to look up all the raw data about job start and finish times. If you want to let this JIRA stay uncommitted until I add in the visual representation, I'm fine with that.
Like Jason already pointed out in our discussion, hiding/showing coloumns will result in "column hell" (I believe he can be credited with coining this term ;-) ) where URLs shared between users might not incorporate the information about which columns are being shown and which are hidden. We could incorporate that into the URL, but I'm guessing there are better ways to present the information. We could also avoid printing the date for jobs which finish on the same day as they started. Like you suggested, we can address that in a separate JIRA.
An "Analyze the job" link in my mind would incorporate something like Vaidya so I'm wary of adding it right now.
I'll file the JIRA to fix the start time.
> Append time elapsed since job-start-time for finished tasks
> -----------------------------------------------------------
>
> Key: MAPREDUCE-4711
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4711
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
> Affects Versions: 0.23.3
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
> Attachments: MAPREDUCE-4711.branch-0.23.patch
>
>
> In 0.20.x/1.x, the analyze job link gave this information
> bq. The last Map task task_<sometask> finished at (relative to the Job launch time): 5/10 20:23:10 (1hrs, 27mins, 54sec)
> The time it took for the last task to finish needs to be calculated mentally in 0.23. I believe we should print it next to the finish time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4711) Append time elapsed since
job-start-time for finished tasks
Posted by "Ravi Prakash (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472493#comment-13472493 ]
Ravi Prakash commented on MAPREDUCE-4711:
-----------------------------------------
My main purpose in filing this JIRA was to present ALL the information that was available in branch-1 in the web UI of 0.23. After this JIRA, for retired jobs, nothing will be missing.
> Append time elapsed since job-start-time for finished tasks
> -----------------------------------------------------------
>
> Key: MAPREDUCE-4711
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4711
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
> Affects Versions: 0.23.3
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
> Attachments: MAPREDUCE-4711.branch-0.23.patch
>
>
> In 0.20.x/1.x, the analyze job link gave this information
> bq. The last Map task task_<sometask> finished at (relative to the Job launch time): 5/10 20:23:10 (1hrs, 27mins, 54sec)
> The time it took for the last task to finish needs to be calculated mentally in 0.23. I believe we should print it next to the finish time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4711) Append time elapsed since
job-start-time for finished tasks
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470648#comment-13470648 ]
Hadoop QA commented on MAPREDUCE-4711:
--------------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548038/MAPREDUCE-4711.branch-0.23.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2909//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2909//console
This message is automatically generated.
> Append time elapsed since job-start-time for finished tasks
> -----------------------------------------------------------
>
> Key: MAPREDUCE-4711
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4711
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
> Affects Versions: 0.23.3
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
> Attachments: MAPREDUCE-4711.branch-0.23.patch
>
>
> In 0.20.x/1.x, the analyze job link gave this information
> bq. The last Map task task_<sometask> finished at (relative to the Job launch time): 5/10 20:23:10 (1hrs, 27mins, 54sec)
> The time it took for the last task to finish needs to be calculated mentally in 0.23. I believe we should print it next to the finish time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira