You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2012/05/23 23:35:41 UTC
[jira] [Created] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Jason Lowe created MAPREDUCE-4283:
-------------------------------------
Summary: Display tail of aggregated logs by default
Key: MAPREDUCE-4283
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobhistoryserver, mrv2
Affects Versions: 0.23.3, 2.0.1-alpha
Reporter: Jason Lowe
Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Status: Open (was: Patch Available)
Canceling patch to address findbug warning.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401611#comment-13401611 ]
Jason Lowe commented on MAPREDUCE-4283:
---------------------------------------
The javadoc warnings are unrelated, see MAPREDUCE-4373.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-4283:
-------------------------------------------
Resolution: Fixed
Fix Version/s: 2.2.0-alpha
3.0.0
0.23.3
Status: Resolved (was: Patch Available)
Thanks Jason,
I put this into trunk, branch-2, and branch-0.23.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3, 3.0.0, 2.2.0-alpha
>
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413146#comment-13413146 ]
Jason Lowe commented on MAPREDUCE-4283:
---------------------------------------
Pinging for review, would like to see this feature integrated soon as we keep running into large aggregated logs and have no good way to view a portion of them without downloading the entire log.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417014#comment-13417014 ]
Hudson commented on MAPREDUCE-4283:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1107 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1107/])
MAPREDUCE-4283. Display tail of aggregated logs by default (Jason Lowe via bobby) (Revision 1362608)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362608
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3, 3.0.0, 2.2.0-alpha
>
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Status: Open (was: Patch Available)
Canceling patch to address review comments.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Attachment: MAPREDUCE-4283.patch
Good catch. Also I noticed there was a bug if we try to skip forward in the log and we encounter a premature EOF. Incorporated fixes for those in an updated patch.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417047#comment-13417047 ]
Hudson commented on MAPREDUCE-4283:
-----------------------------------
Integrated in Hadoop-Hdfs-0.23-Build #317 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/317/])
svn merge -c 1362608 FIXES: MAPREDUCE-4283. Display tail of aggregated logs by default (Jason Lowe via bobby) (Revision 1362610)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362610
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3, 3.0.0, 2.2.0-alpha
>
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Siddharth Seth (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414823#comment-13414823 ]
Siddharth Seth commented on MAPREDUCE-4283:
-------------------------------------------
Overall, lgtm. Minor nits.
- AggregatedLogFormat.getContainerLogsReader should accept a ContainerId instead of a String AggLogFormat should be responsible for converting the ContainerId to whatever format it uses.
- If there's nothing to read (start == end) - AggregatedLogsBlock.readContainerLogs closes an unopened pre block.
Also, I think AggregatedLogsFormat should be Private / Unstable for now. Will create a separate jira.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416505#comment-13416505 ]
Hudson commented on MAPREDUCE-4283:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2554 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2554/])
MAPREDUCE-4283. Display tail of aggregated logs by default (Jason Lowe via bobby) (Revision 1362608)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362608
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3, 3.0.0, 2.2.0-alpha
>
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Status: Patch Available (was: Open)
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415382#comment-13415382 ]
Hadoop QA commented on MAPREDUCE-4283:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12536652/MAPREDUCE-4283.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2598//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2598//console
This message is automatically generated.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414112#comment-13414112 ]
Hadoop QA commented on MAPREDUCE-4283:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12536467/MAPREDUCE-4283.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2589//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2589//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2589//console
This message is automatically generated.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417107#comment-13417107 ]
Hudson commented on MAPREDUCE-4283:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1140 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1140/])
MAPREDUCE-4283. Display tail of aggregated logs by default (Jason Lowe via bobby) (Revision 1362608)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362608
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3, 3.0.0, 2.2.0-alpha
>
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Attachment: MAPREDUCE-4283.patch
Patch to add start/end parameters to aggregated log pages and uses start=-4K by default. Manually tested by running a sleep job on a single-node cluster with compressed aggregated logs enabled and examining the logs from the history server web pages.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Attachments: MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415506#comment-13415506 ]
Hadoop QA commented on MAPREDUCE-4283:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12536688/MAPREDUCE-4283.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2601//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2601//console
This message is automatically generated.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13401511#comment-13401511 ]
Hadoop QA commented on MAPREDUCE-4283:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533497/MAPREDUCE-4283.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 javadoc. The javadoc tool appears to have generated 2 warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2516//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2516//console
This message is automatically generated.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414143#comment-13414143 ]
Hadoop QA commented on MAPREDUCE-4283:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12536472/MAPREDUCE-4283.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2590//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2590//console
This message is automatically generated.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415228#comment-13415228 ]
Robert Joseph Evans commented on MAPREDUCE-4283:
------------------------------------------------
I like the changes, but I have one new nit with it. Inside getContainerLogsReader I don't like using toString for the comparison of the key and the containerId. I know that is what has to ultimately happen, but I would prefer to see that code move up into LogKey. I would like to see a method like boolean LogKey.isFor(ContainerId). LogKey holds the code that knows how to initialize itself from a ContainerId, I would prefer to have it also hold the code so it knows how to check to see if they are the same.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415476#comment-13415476 ]
Robert Joseph Evans commented on MAPREDUCE-4283:
------------------------------------------------
I am +1 on this patch, but I am going to wait to check it in until Sid has a chance to look at it too.
I don't want to kill this patch with minor nits, so I am OK with this going in as is. If you want to fix it feel free. But LogKey hash and equals are inconsistent. In hash you handle the case of keyString == null, but in equals you do not. You will get an NPE if it is null. It should never be null, because other places will throw NPEs too, so it would be best just to check for null in the constructor.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Attachment: MAPREDUCE-4283.patch
Thanks for the review Sidd!
I changed AggregatedLogFormat.getContainerLogsReader to take a ContainerId instead of a String.
I don't think there's an unbalanced pre block issue with start == end, as the pre block is unconditionally opened at this line:
{code}
PRE<Hamlet> pre = html.pre();
{code}
and then unconditionally closed after the while read loop. I manually tested it with start == end, and it created a degenerate pre block: {{<pre></pre>}}
Didn't see a JIRA go by about making AggregatedLogFormat Private/Unstable, so I filed MAPREDUCE-4446 to track that issue.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416542#comment-13416542 ]
Hudson commented on MAPREDUCE-4283:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2510 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2510/])
MAPREDUCE-4283. Display tail of aggregated logs by default (Jason Lowe via bobby) (Revision 1362608)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362608
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3, 3.0.0, 2.2.0-alpha
>
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Assignee: Jason Lowe
Target Version/s: 0.23.3, 2.0.1-alpha
Status: Patch Available (was: Open)
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-4283) Display tail of aggregated logs by
default
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-4283.
------------------------------------
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3, 2.0.2-alpha
>
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Attachment: MAPREDUCE-4283.patch
New patch with equals fix.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Status: Patch Available (was: Open)
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416502#comment-13416502 ]
Hudson commented on MAPREDUCE-4283:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2489 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2489/])
MAPREDUCE-4283. Display tail of aggregated logs by default (Jason Lowe via bobby) (Revision 1362608)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362608
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/TestHSWebApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/AggregatedLogFormat.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/log/AggregatedLogsBlock.java
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Fix For: 0.23.3, 3.0.0, 2.2.0-alpha
>
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Attachment: MAPREDUCE-4283.patch
Rather than add an isFor method, I went ahead and added hashCode and equals methods to LogKey so we can compare keys directly.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Jason Lowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated MAPREDUCE-4283:
----------------------------------
Attachment: MAPREDUCE-4283.patch
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415217#comment-13415217 ]
Hadoop QA commented on MAPREDUCE-4283:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12536644/MAPREDUCE-4283.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-common.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2597//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2597//console
This message is automatically generated.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416455#comment-13416455 ]
Robert Joseph Evans commented on MAPREDUCE-4283:
------------------------------------------------
It has been a day with no new comments. Sid I am going to check this in. If you do have any follow up issues for this we can do them on a separate JIRA.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.1.0-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch, MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4283) Display tail of aggregated logs
by default
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414011#comment-13414011 ]
Robert Joseph Evans commented on MAPREDUCE-4283:
------------------------------------------------
Jason,
There are some issues with how you seek in the file.
InputStream.available is only supposed to return the value that is left and will not block. It looks like you are using it to try and read to the end of the BoundedInputStream, but I am not sure that it is guaranteed to work that way.
> Display tail of aggregated logs by default
> ------------------------------------------
>
> Key: MAPREDUCE-4283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver, mrv2
> Affects Versions: 0.23.3, 2.0.1-alpha
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Attachments: MAPREDUCE-4283.patch
>
>
> Similar to the manner in which the nodemanager webUI displays container logs, it would be very useful if the historyserver showed the trailing 4K or so of the aggregated logs with a link to see the full log.
> When debugging issues the relevant errors are usually at the end of the log, so showing just the last few K can enable quick diagnosis without waiting for what can be many megabytes of log data to download.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira