You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Arun C Murthy (JIRA)" <ji...@apache.org> on 2009/10/16 19:56:31 UTC
[jira] Created: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Serialize the 'final'ness of Configuration keys
-----------------------------------------------
Key: HADOOP-6317
URL: https://issues.apache.org/jira/browse/HADOOP-6317
Project: Hadoop Common
Issue Type: Improvement
Components: conf
Reporter: Arun C Murthy
Fix For: 0.22.0
Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
# TaskTracker should mark the tasks' 'localized' parameters as 'final'
# GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787671#action_12787671 ]
Aaron Kimball commented on HADOOP-6317:
---------------------------------------
Arun,
Is this patch where you'd like it to be? Anything left to do here?
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball reassigned HADOOP-6317:
-------------------------------------
Assignee: Aaron Kimball
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768339#action_12768339 ]
Arun C Murthy commented on HADOOP-6317:
---------------------------------------
I was thinking along the lines of the tasktracker doing something like:
conf.setFinalInt("mapred.task.id", taskAttemptId);
or
conf.setInt("mapred.task.id", taskAttemptId, true);
where we have the necessary apis like so:
Configuration.setFinalInt(String key, int value);
or
Configuration.setInt(String key, int value, boolean finalParam);
Does that make sense?
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-6317:
---------------------------------
Attachment: HADOOP-6317.patch
Fixed patch to apply to current trunk.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "gary murry (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770658#action_12770658 ]
gary murry commented on HADOOP-6317:
------------------------------------
{quote}I can either remove the try..catch block from the method, or we can up the acceptable findbugs warning count. Someone else want to weigh in?{quote}
It would be best not to up the aceptable findbugs warning count.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Status: Patch Available (was: Open)
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-6317:
----------------------------------
Status: Open (was: Patch Available)
Doug, the patch looks good. I propose we mark the new apis @Private to let end-users know that this is only for the framework (TTs etc.). Thoughts?
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Attachment: HADOOP-6317.patch
Attaching patch which meets this issue. Includes a test case which verifies both positive and negative cases for serializing {{<final>true</final>}} for a given property.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767460#action_12767460 ]
Aaron Kimball commented on HADOOP-6317:
---------------------------------------
Arun,
Do you mean, that all keys in a Configuration object should be added to its {{finalParameters}} set?
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Status: Open (was: Patch Available)
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Status: Patch Available (was: Open)
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Attachment: HADOOP-6317.3.patch
New patch, fixes javadoc warning. findbugs warning is same as before.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-6317:
---------------------------------
Status: Open (was: Patch Available)
Found this in the patch queue. The patch no longer applies cleanly. I'll try to repair it.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Attachment: HADOOP-6317.2.patch
Arun, here's a new implementation with your suggestion incorporated. Adds a test case too.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Status: Patch Available (was: Open)
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Attachment: HADOOP-6317.4.patch
More specific {{catch}} block to eliminate findbugs warning.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924014#action_12924014 ]
Hadoop QA commented on HADOOP-6317:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12457858/HADOOP-6317.patch
against trunk revision 1022697.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system tests framework. The patch passed system tests framework compile.
Test results: http://hudson.zones.apache.org/hudson/job/PreCommit-HADOOP-Build/4/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/PreCommit-HADOOP-Build/4/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/PreCommit-HADOOP-Build/4/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/PreCommit-HADOOP-Build/4/console
This message is automatically generated.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767428#action_12767428 ]
Aaron Kimball commented on HADOOP-6317:
---------------------------------------
The "new" findbugs warning refers to the {{try .. catch (Exception)}} block in {{writeXml()}}. This shouldn't be new; the lines I added to the method shouldn't affect the utility of this try block. Nevertheless, findbugs seems to be picking this up for the first time now.
I can either remove the try..catch block from the method, or we can up the acceptable findbugs warning count. Someone else want to weigh in?
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924781#action_12924781 ]
Hadoop QA commented on HADOOP-6317:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12457858/HADOOP-6317.patch
against trunk revision 1027301.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/8//console
This message is automatically generated.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924799#action_12924799 ]
Hadoop QA commented on HADOOP-6317:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12457858/HADOOP-6317.patch
against trunk revision 1027316.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system tests framework. The patch passed system tests framework compile.
Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/10//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/10//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/10//artifact/trunk/build/test/checkstyle-errors.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/10//console
This message is automatically generated.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767898#action_12767898 ]
Aaron Kimball commented on HADOOP-6317:
---------------------------------------
Arun,
Let me know if you mean something like this:
{code}
/**
* Marks a specific key in the Configuration as final.
*/
public void makeFinal(String key) {
if (!finalParameters.contains(key)) {
finalParameters.add(key);
}
}
/**
* Marks all keys in the Configuration as final.
*/
public void makeConfigurationFinal() {
Properties properties = getProps();
for (Enumeration<String> e = properties.keys(); e.hasMoreElements();) {
String key = e.nextElement();
makeFinal(key);
}
}
{code}
if so, I'll add this and a unit test to the patch.
- Aaron
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Status: Open (was: Patch Available)
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768488#action_12768488 ]
Hadoop QA commented on HADOOP-6317:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12422853/HADOOP-6317.2.patch
against trunk revision 828181.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/97/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/97/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/97/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/97/console
This message is automatically generated.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-6317:
---------------------------------
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
{code}
-1 overall.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 system tests framework. The patch passed system tests framework compile.
{code}
I don't get the javadoc failure, since there are the same 6 javadoc warnings with or without this patch.
Arun, do you have concerns with this patch? If not, I'll commit it.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768497#action_12768497 ]
Hadoop QA commented on HADOOP-6317:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12422856/HADOOP-6317.3.patch
against trunk revision 828181.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/98/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/98/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/98/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/98/console
This message is automatically generated.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Status: Open (was: Patch Available)
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6317:
----------------------------------
Status: Patch Available (was: Open)
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770681#action_12770681 ]
Hadoop QA commented on HADOOP-6317:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12423366/HADOOP-6317.4.patch
against trunk revision 829289.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/106/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/106/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/106/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/106/console
This message is automatically generated.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.2.patch, HADOOP-6317.3.patch, HADOOP-6317.4.patch, HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12767441#action_12767441 ]
Arun C Murthy commented on HADOOP-6317:
---------------------------------------
Aaron - Thinking a bit more about this... one important feature is the ability for the TaskTracker to 'mark' configs as final programmatically. For e.g. the localized configs should be marked final. Does that make sense? Apologies for missing this in the original proposal...
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6317) Serialize the 'final'ness of
Configuration keys
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766830#action_12766830 ]
Hadoop QA commented on HADOOP-6317:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12422427/HADOOP-6317.patch
against trunk revision 826141.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/91/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/91/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/91/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/91/console
This message is automatically generated.
> Serialize the 'final'ness of Configuration keys
> -----------------------------------------------
>
> Key: HADOOP-6317
> URL: https://issues.apache.org/jira/browse/HADOOP-6317
> Project: Hadoop Common
> Issue Type: Improvement
> Components: conf
> Reporter: Arun C Murthy
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: HADOOP-6317.patch
>
>
> Currently Configuration.writeXml doesn't serialize the 'final' attribute. There are some scenarios where it is useful:
> # TaskTracker should mark the tasks' 'localized' parameters as 'final'
> # GUI tools
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.