You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Hemanth Yamijala (JIRA)" <ji...@apache.org> on 2009/09/01 13:30:33 UTC
[jira] Created: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Configuration does not lock parameters marked final if they have no value.
--------------------------------------------------------------------------
Key: HADOOP-6227
URL: https://issues.apache.org/jira/browse/HADOOP-6227
Project: Hadoop Common
Issue Type: Bug
Components: conf
Reporter: Hemanth Yamijala
A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-6227:
--------------------------------------------
Attachment: patch-6227.txt
patch-6227-0.20.txt
Patch with the fix
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Fix For: 0.20.1
>
> Attachments: patch-6227-0.20.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-6227:
------------------------------------
Release Note: (was: Modified configuration to allow setting final parameters with null values.)
Editorial pass over all release notes prior to publication of 0.21. Bug.
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-6227-0.20.txt, patch-6227-ydist.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hemanth Yamijala updated HADOOP-6227:
-------------------------------------
Resolution: Fixed
Fix Version/s: (was: 0.20.1)
0.21.0
Assignee: Amareshwari Sriramadasu
Release Note: Modified configuration to allow setting final parameters with null values.
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this to trunk. Thanks, Amareshwari !
I did not commit to the 0.20 branch, because there's an active vote in progress and did not think this existing bug should block the release. If decided otherwise, we can commit to the 0.20 branch as well.
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-6227-0.20.txt, patch-6227-ydist.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749949#action_12749949 ]
Hudson commented on HADOOP-6227:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #4 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/4/])
. Fix Configuration to allow final parameters to be set to null and prevent them from being overridden. Contributed by Amareshwari Sriramadasu.
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-6227-0.20.txt, patch-6227-ydist.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12750389#action_12750389 ]
Hudson commented on HADOOP-6227:
--------------------------------
Integrated in Hadoop-Common-trunk #82 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/82/])
. Fix Configuration to allow final parameters to be set to null and prevent them from being overridden. Contributed by Amareshwari Sriramadasu.
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-6227-0.20.txt, patch-6227-ydist.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749914#action_12749914 ]
Vinod K V commented on HADOOP-6227:
-----------------------------------
I just tested this fix on a single node cluster so as to enable/disable mapred debug script by setting it to null. The patch works as expected. A null value marked as final on server side can no longer be overridden by clients.
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Fix For: 0.20.1
>
> Attachments: patch-6227-0.20.txt, patch-6227-ydist.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-6227:
--------------------------------------------
Fix Version/s: 0.20.1
Status: Patch Available (was: Open)
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Fix For: 0.20.1
>
> Attachments: patch-6227-0.20.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749892#action_12749892 ]
Hadoop QA commented on HADOOP-6227:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12418238/patch-6227.txt
against trunk revision 809491.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/4/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/4/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/4/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/4/console
This message is automatically generated.
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Fix For: 0.20.1
>
> Attachments: patch-6227-0.20.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vinod K V updated HADOOP-6227:
------------------------------
Attachment: patch-6227-ydist.txt
Yahoo! distribution patch. The patch for branch 0.20 will not apply to this distribution with HADOOP-6184 already in.
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Fix For: 0.20.1
>
> Attachments: patch-6227-0.20.txt, patch-6227-ydist.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749907#action_12749907 ]
Hemanth Yamijala commented on HADOOP-6227:
------------------------------------------
Code changes look fine to me. +1.
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Fix For: 0.20.1
>
> Attachments: patch-6227-0.20.txt, patch-6227-ydist.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Hemanth Yamijala (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749866#action_12749866 ]
Hemanth Yamijala commented on HADOOP-6227:
------------------------------------------
The code in configuration looks like this:
{code}
// Ignore this parameter if it has already been marked as 'final'
if (attr != null && value != null) {
if (!finalParameters.contains(attr)) {
properties.setProperty(attr, value);
if (storeResource) {
updatingResource.put(attr, name.toString());
}
if (finalParameter)
finalParameters.add(attr);
} else {
LOG.warn(name+":a attempt to override final parameter: "+attr
+"; Ignoring.");
}
}
{code}
While the check for value being null is required before adding the property to the 'properties' instance, adding it to finalParameters or for that matter even updatingResource shouldn't be an issue even if the value is null. This would certainly fix the current issue. Thoughts ?
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6227) Configuration does not lock
parameters marked final if they have no value.
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749920#action_12749920 ]
Arun C Murthy commented on HADOOP-6227:
---------------------------------------
+1
> Configuration does not lock parameters marked final if they have no value.
> --------------------------------------------------------------------------
>
> Key: HADOOP-6227
> URL: https://issues.apache.org/jira/browse/HADOOP-6227
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Hemanth Yamijala
> Fix For: 0.20.1
>
> Attachments: patch-6227-0.20.txt, patch-6227-ydist.txt, patch-6227.txt
>
>
> A use-case: Recently, we stumbled on a bug that wanted us to disable the feature to run a debug script in map/reduce on tasks that fail, specified by mapred.{map|reduce}.task.debug.script. The best way of disabling it seemed to be to set it with no value in the cluster configuration and mark it final. This did not work, however, because the code in configuration explicitly checks if the value is not null before adding it to the list of final parameters.
> Without an ability to do this, there might be a need to explicitly have a special way of disabling optional features.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.