You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Shevek (JIRA)" <ji...@apache.org> on 2009/04/24 21:28:30 UTC
[jira] Created: (HADOOP-5743) Configuration: Hierachical;
documented; null-safe; thread-safe
Configuration: Hierachical; documented; null-safe; thread-safe
--------------------------------------------------------------
Key: HADOOP-5743
URL: https://issues.apache.org/jira/browse/HADOOP-5743
Project: Hadoop Core
Issue Type: Improvement
Components: conf
Reporter: Shevek
Patch to configuration to:
* Update documentation to match code
* Make configurations hierarchical
* Remove global REGISTRY of configurations
* Adding global default does not reload child configurations
* (Eventually) make properly thread-safe
* Allow well-behaved overriding with nulls
* Allow future satisfaction of ticket 5708 (with which I agree, but Doug does not).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5743) Configuration: Hierachical;
documented; null-safe; thread-safe
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702568#action_12702568 ]
Doug Cutting commented on HADOOP-5743:
--------------------------------------
I don't know all of the places where Configuration is used as a Writable, but my hunch is that it's okay to change its format. Where it's passed in an RPC, an incompatible change is okay, since we don't permit cross-version RPC. Configurations are written to disk on the job tracker, but I don't think we guarantee that the job queue survives upgrades.
It might be nice is if it included a version number in the serialized form. The new version could be -1, so that old code reading the new format would see this as the size and fail. But new code reading the old could do something back-compatible when the version is positive.
There are still lots of whitespace-only changes in your patch that inject noise and make me stop reading it.
I'm also not (yet) highly incented to spend a lot of time reviewing this, since it's not entirely clear to me that this is fixing problems that folks are actually encountering. Radically changing something low-level like this has a very good chance of breaking lots of systems, which will take a lot of time to debug and fix, and should thus not be entered into lightly.
That said, if you feel strongly, I encourage you to develop this to the point where it passes unit tests and can be automatically tested by Hudson. Also, please read your patch files before you submit them to see that they only include changes that are critical to your mission. They should not include discussions of the changes annotated with your name. Jira is for discussion of patches.
http://wiki.apache.org/hadoop/HowToContribute
> Configuration: Hierachical; documented; null-safe; thread-safe
> --------------------------------------------------------------
>
> Key: HADOOP-5743
> URL: https://issues.apache.org/jira/browse/HADOOP-5743
> Project: Hadoop Core
> Issue Type: Improvement
> Components: conf
> Reporter: Shevek
> Attachments: 01_configuration.patch, 02_configuration.patch
>
>
> Patch to configuration to:
> * Update documentation to match code
> * Make configurations hierarchical
> * Remove global REGISTRY of configurations
> * Adding global default does not reload child configurations
> * (Eventually) make properly thread-safe
> * Allow well-behaved overriding with nulls
> * Allow future satisfaction of ticket 5708 (with which I agree, but Doug does not).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5743) Configuration: Hierachical;
documented; null-safe; thread-safe
Posted by "Shevek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702495#action_12702495 ]
Shevek commented on HADOOP-5743:
--------------------------------
Doug, is the configuration serialization format allowed to change (to allow on-wire preservation of hierarchy metadata, and thus runtime semantics of resources/null-override-safety/etc), or would that be a disallowed incompatibility?
> Configuration: Hierachical; documented; null-safe; thread-safe
> --------------------------------------------------------------
>
> Key: HADOOP-5743
> URL: https://issues.apache.org/jira/browse/HADOOP-5743
> Project: Hadoop Core
> Issue Type: Improvement
> Components: conf
> Reporter: Shevek
> Attachments: 01_configuration.patch
>
>
> Patch to configuration to:
> * Update documentation to match code
> * Make configurations hierarchical
> * Remove global REGISTRY of configurations
> * Adding global default does not reload child configurations
> * (Eventually) make properly thread-safe
> * Allow well-behaved overriding with nulls
> * Allow future satisfaction of ticket 5708 (with which I agree, but Doug does not).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5743) Configuration: Hierachical;
documented; null-safe; thread-safe
Posted by "Shevek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shevek updated HADOOP-5743:
---------------------------
Release Note: Second draft will satisfy comments in 5708, but leaves questions outstanding. (was: Patch attached is first draft, second draft will satisfy comments in 5708.)
Status: Patch Available (was: Open)
> Configuration: Hierachical; documented; null-safe; thread-safe
> --------------------------------------------------------------
>
> Key: HADOOP-5743
> URL: https://issues.apache.org/jira/browse/HADOOP-5743
> Project: Hadoop Core
> Issue Type: Improvement
> Components: conf
> Reporter: Shevek
> Attachments: 01_configuration.patch, 02_configuration.patch
>
>
> Patch to configuration to:
> * Update documentation to match code
> * Make configurations hierarchical
> * Remove global REGISTRY of configurations
> * Adding global default does not reload child configurations
> * (Eventually) make properly thread-safe
> * Allow well-behaved overriding with nulls
> * Allow future satisfaction of ticket 5708 (with which I agree, but Doug does not).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5743) Configuration: Hierachical;
documented; null-safe; thread-safe
Posted by "Shevek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shevek updated HADOOP-5743:
---------------------------
Attachment: 02_configuration.patch
Change to wire format would also allow preservation of finalParameters, which is no longer broken with this patch.
Kinda needs review. Lots of questions in the code. A better synchronization structure rather requires a slightly different data model, where finalParameters is stored within the Map.
> Configuration: Hierachical; documented; null-safe; thread-safe
> --------------------------------------------------------------
>
> Key: HADOOP-5743
> URL: https://issues.apache.org/jira/browse/HADOOP-5743
> Project: Hadoop Core
> Issue Type: Improvement
> Components: conf
> Reporter: Shevek
> Attachments: 01_configuration.patch, 02_configuration.patch
>
>
> Patch to configuration to:
> * Update documentation to match code
> * Make configurations hierarchical
> * Remove global REGISTRY of configurations
> * Adding global default does not reload child configurations
> * (Eventually) make properly thread-safe
> * Allow well-behaved overriding with nulls
> * Allow future satisfaction of ticket 5708 (with which I agree, but Doug does not).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5743) Configuration: Hierachical;
documented; null-safe; thread-safe
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-5743:
--------------------------------
Assignee: Shevek
Status: Open (was: Patch Available)
This clearly is under discussion. Moving the issue to Open state.
> Configuration: Hierachical; documented; null-safe; thread-safe
> --------------------------------------------------------------
>
> Key: HADOOP-5743
> URL: https://issues.apache.org/jira/browse/HADOOP-5743
> Project: Hadoop Core
> Issue Type: Improvement
> Components: conf
> Reporter: Shevek
> Assignee: Shevek
> Attachments: 01_configuration.patch, 02_configuration.patch
>
>
> Patch to configuration to:
> * Update documentation to match code
> * Make configurations hierarchical
> * Remove global REGISTRY of configurations
> * Adding global default does not reload child configurations
> * (Eventually) make properly thread-safe
> * Allow well-behaved overriding with nulls
> * Allow future satisfaction of ticket 5708 (with which I agree, but Doug does not).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5743) Configuration: Hierachical;
documented; null-safe; thread-safe
Posted by "Shevek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shevek updated HADOOP-5743:
---------------------------
Attachment: 01_configuration.patch
First draft patch, comments from Doug in 5708 outstanding.
> Configuration: Hierachical; documented; null-safe; thread-safe
> --------------------------------------------------------------
>
> Key: HADOOP-5743
> URL: https://issues.apache.org/jira/browse/HADOOP-5743
> Project: Hadoop Core
> Issue Type: Improvement
> Components: conf
> Reporter: Shevek
> Attachments: 01_configuration.patch
>
>
> Patch to configuration to:
> * Update documentation to match code
> * Make configurations hierarchical
> * Remove global REGISTRY of configurations
> * Adding global default does not reload child configurations
> * (Eventually) make properly thread-safe
> * Allow well-behaved overriding with nulls
> * Allow future satisfaction of ticket 5708 (with which I agree, but Doug does not).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5743) Configuration: Hierachical;
documented; null-safe; thread-safe
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702867#action_12702867 ]
Hadoop QA commented on HADOOP-5743:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12406396/02_configuration.patch
against trunk revision 768376.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 2 new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/245/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/245/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/245/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/245/console
This message is automatically generated.
> Configuration: Hierachical; documented; null-safe; thread-safe
> --------------------------------------------------------------
>
> Key: HADOOP-5743
> URL: https://issues.apache.org/jira/browse/HADOOP-5743
> Project: Hadoop Core
> Issue Type: Improvement
> Components: conf
> Reporter: Shevek
> Attachments: 01_configuration.patch, 02_configuration.patch
>
>
> Patch to configuration to:
> * Update documentation to match code
> * Make configurations hierarchical
> * Remove global REGISTRY of configurations
> * Adding global default does not reload child configurations
> * (Eventually) make properly thread-safe
> * Allow well-behaved overriding with nulls
> * Allow future satisfaction of ticket 5708 (with which I agree, but Doug does not).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.