You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Karthik Kambatla (JIRA)" <ji...@apache.org> on 2012/09/01 04:37:07 UTC
[jira] [Created] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Karthik Kambatla created MAPREDUCE-4629:
-------------------------------------------
Summary: In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
Key: MAPREDUCE-4629
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 1.0.3
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4629:
----------------------------------------
Attachment: MR-4629.patch
Uploading a patch that fixes this issue.
The patch:
- corrects the adjustment in directoryTime() and timestamDirectoryComponent() for DEBUG_MODE
- adds tests to ensure correct behavior
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4629:
----------------------------------------
Attachment: MR-4629.patch.v1
Uploading a preliminary patch. The patch -
- adds hour/min fields to the file names in DEBUG_MODE
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch, MR-4629.patch.v1
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450173#comment-13450173 ]
Karthik Kambatla commented on MAPREDUCE-4629:
---------------------------------------------
Modified this JIRA to remove the DEBUG_MODE. Created https://issues.apache.org/jira/browse/MAPREDUCE-4643 to make the cleanup period configurable.
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch, MR-4629.patch.v1
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450785#comment-13450785 ]
Hudson commented on MAPREDUCE-4629:
-----------------------------------
Integrated in Hadoop-Common-trunk-Commit #2706 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2706/])
MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby) (Revision 1382088)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382088
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 1.2.0, 2.2.0-alpha
>
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4629:
----------------------------------------
Status: Patch Available (was: In Progress)
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.0.0-alpha, 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451337#comment-13451337 ]
Hudson commented on MAPREDUCE-4629:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #1190 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1190/])
MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby) (Revision 1382088)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382088
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 1.2.0, 2.0.2-alpha
>
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450089#comment-13450089 ]
Karthik Kambatla commented on MAPREDUCE-4629:
---------------------------------------------
Robert and Alejandro,
Thanks for your comments. Given your inputs, the best way forward might be to (1) remove DEBUG_MODE in this JIRA and use LOG.debug() and LOG.trace() where needed, and (2) create a separate JIRA for adding configurable history-cleanup-time.
If you think that makes sense, I ll go ahead with it.
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch, MR-4629.patch.v1
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4629:
----------------------------------------
Status: Patch Available (was: In Progress)
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-4629:
-------------------------------------------
Attachment: trunk-mr-6429.patch
Uploading the trunk patch again, unchanged to Jenkins can run the precommit build on it.
The changes look good to me. Assuming Jenkins comes back OK I am a +1 on the changes for trunk. Assuming my running the unit tests on branhc-1 come back OK I am +1 for that as well.
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446583#comment-13446583 ]
Hadoop QA commented on MAPREDUCE-4629:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12543406/MR-4629.patch
against trunk revision .
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2809//console
This message is automatically generated.
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450783#comment-13450783 ]
Karthik Kambatla commented on MAPREDUCE-4629:
---------------------------------------------
Thanks a lot, Bobby.
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 1.2.0, 2.2.0-alpha
>
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4629:
----------------------------------------
Attachment: trunk-mr-6429.patch
Uploading a patch for trunk that removes debug mode. I haven't added/removed any tests because there weren't any debug mode tests earlier.
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447886#comment-13447886 ]
Karthik Kambatla commented on MAPREDUCE-4629:
---------------------------------------------
>From my conversation with Alejandro offline:
More context:
- The regular mode cleans up the history files after a month. The history filenames use yyyy/mm/dd.
- The debug mode cleans up the history files after 20 minutes. The history filenames (currently) use yyyy/hour/min.
The DEBUG_MODE overloads the regular mode. Instead, a better approach seems to be to append to the regular mode, instead of overloading it.
Also, the config parameter to turn the DEBUG_MODE on/off mapreduce.jobhistory.debug.mode doesn't have a default in mapred-default.xml and needs to be added.
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449867#comment-13449867 ]
Karthik Kambatla commented on MAPREDUCE-4629:
---------------------------------------------
The DEBUG_MODE seems to be doing a lot of things. It has a shorter cleanup time for history files, history filenames use different timestamps, prints stack traces at various points in the code.
Given that the config option was not present in the mapred-default.xml so far, I wonder if anyone is aware of and using DEBUG_MODE at all. One option is to remove the DEBUG_MODE and do the following instead:
- Add a config-parameter for history cleanup
- Replace instances of debug-printing with LOG.debug()
- To address the ability to verify shorter cleanup times, add the hour/min to the history filenames.
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch, MR-4629.patch.v1
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450071#comment-13450071 ]
Alejandro Abdelnur commented on MAPREDUCE-4629:
-----------------------------------------------
Karthik,
If DEBUG_MODE is there only for the purposes of reducing the hardcoded cleanup time, then I agree we should make this configurable. Default value should be the current 'non-debug' max age of 1 month. We should have a consistent directory layout regardless of the cleanup time, I'd suggest using yyyy/mm/dd-HH-MM, to avoid having a subdir explosion. the cleanup logic should be smart enough to look at the necessary granularity in the name. Regarding the printing of stack traces, those should be converted to LOG.trace() calls. Make sense?
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch, MR-4629.patch.v1
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4629:
----------------------------------------
Attachment: branch1-mr-4629.patch
Uploading a patch for branch1.
TestJobHistory* have passed but for two errors in TestJobHistoryConfig. The erroneous tests had errors in branch1 also.
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450070#comment-13450070 ]
Robert Joseph Evans commented on MAPREDUCE-4629:
------------------------------------------------
DEBUG_MODE for the history server is present also in trunk and 2.0 as well.
I don't believe that anyone is using it, but there could be a few people using it to test the history server outside of Hadoop proper. I am +1 for removing it. I don't see much of a reason to change anything at this point except to rip it out, and amybe make the debug-printing LOG.debug. If someone else wants similar functionality for testing we can add in targeted configs to do things like what Karthik is suggesting.
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch, MR-4629.patch.v1
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy closed MAPREDUCE-4629.
------------------------------------
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 1.2.0, 2.0.2-alpha
>
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on MAPREDUCE-4629 started by Karthik Kambatla.
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on MAPREDUCE-4629 started by Karthik Kambatla.
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4629:
----------------------------------------
Description:
Remove JobHistory.DEBUG_MODE for the following reasons:
1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
was:
The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
Affects Version/s: 2.0.0-alpha
Hadoop Flags: Incompatible change
Summary: Remove JobHistory.DEBUG_MODE (was: In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time)
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch, MR-4629.patch.v1
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451330#comment-13451330 ]
Hudson commented on MAPREDUCE-4629:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk #1159 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1159/])
MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby) (Revision 1382088)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382088
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 1.2.0, 2.0.2-alpha
>
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450787#comment-13450787 ]
Hudson commented on MAPREDUCE-4629:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #2769 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2769/])
MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby) (Revision 1382088)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382088
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 1.2.0, 2.2.0-alpha
>
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450432#comment-13450432 ]
Hadoop QA commented on MAPREDUCE-4629:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12544175/branch1-mr-4629.patch
against trunk revision .
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2830//console
This message is automatically generated.
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450786#comment-13450786 ]
Hudson commented on MAPREDUCE-4629:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #2730 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2730/])
MAPREDUCE-4629. Remove JobHistory.DEBUG_MODE (Karthik Kambatla via bobby) (Revision 1382088)
Result = FAILURE
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1382088
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 1.2.0, 2.2.0-alpha
>
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450775#comment-13450775 ]
Robert Joseph Evans commented on MAPREDUCE-4629:
------------------------------------------------
Jenkis came back OK and the 1.0 tests for me passed. I'll check this in. Thanks for the patch this is something I have wanted to see happen for a while.
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) In DEBUG_MODE,
JobHistory#directoryTime() returns incorrect time
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karthik Kambatla updated MAPREDUCE-4629:
----------------------------------------
Status: Open (was: Patch Available)
> In DEBUG_MODE, JobHistory#directoryTime() returns incorrect time
> ----------------------------------------------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: MR-4629.patch
>
>
> The helper methods in JobHistory - directoryTime() and timestampDirectoryComponent() - adjust the month field for readability (Jan is 0 as per Calendar, 1 for us)
> In DEBUG_MODE, JobHistory uses hour instead of month. However, the adjustment is still applied.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Robert Joseph Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated MAPREDUCE-4629:
-------------------------------------------
Resolution: Fixed
Fix Version/s: 2.2.0-alpha
1.2.0
Status: Resolved (was: Patch Available)
Thanks Karthik,
I checked this into trunk, branch-2, and branch-1
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Fix For: 1.2.0, 2.2.0-alpha
>
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4629) Remove JobHistory.DEBUG_MODE
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450765#comment-13450765 ]
Hadoop QA commented on MAPREDUCE-4629:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12544233/trunk-mr-6429.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2831//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2831//console
This message is automatically generated.
> Remove JobHistory.DEBUG_MODE
> ----------------------------
>
> Key: MAPREDUCE-4629
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4629
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1.0.3, 2.0.0-alpha
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
> Attachments: branch1-mr-4629.patch, MR-4629.patch, MR-4629.patch.v1, trunk-mr-6429.patch, trunk-mr-6429.patch
>
>
> Remove JobHistory.DEBUG_MODE for the following reasons:
> 1. No one seems to be using it - the config parameter corresponding to enabling it does not even exist in mapred-default.xml
> 2. The logging being done in DEBUG_MODE needs to move to LOG.debug() and LOG.trace()
> 3. Buggy handling of helper methods in DEBUG_MODE; e.g. directoryTime() and timestampDirectoryComponent().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira