You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Patrick Angeles (JIRA)" <ji...@apache.org> on 2011/01/25 18:48:44 UTC
[jira] Created: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
------------------------------------------------------------------------------------
Key: HDFS-1596
URL: https://issues.apache.org/jira/browse/HDFS-1596
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Reporter: Patrick Angeles
The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
<property>
<name>fs.checkpoint.dir</name>
<value>${hadoop.tmp.dir}/dfs/namesecondary</value>
<description>Determines where on the local filesystem the DFS secondary
name node should store the temporary images to merge.
If this is a comma-delimited list of directories then the image is
replicated in all of the directories for redundancy.
</description>
</property>
<property>
<name>fs.checkpoint.edits.dir</name>
<value>${fs.checkpoint.dir}</value>
<description>Determines where on the local filesystem the DFS secondary
name node should store the temporary edits to merge.
If this is a comma-delimited list of directoires then teh edits is
replicated in all of the directoires for redundancy.
Default value is same as fs.checkpoint.dir
</description>
</property>
<property>
<name>fs.checkpoint.period</name>
<value>3600</value>
<description>The number of seconds between two periodic checkpoints.
</description>
</property>
<property>
<name>fs.checkpoint.size</name>
<value>67108864</value>
<description>The size of the current edit log (in bytes) that triggers
a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
</description>
</property>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1596:
-----------------------------------------
Component/s: documentation
Hadoop Flags: [Reviewed]
> I have not a clue why these test cases started to fail with this patch. ...
This is now just documentation changes. The failed tests are not related.
+1 patch looks good.
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated HDFS-1596:
------------------------------------
Status: Patch Available (was: Open)
Re-marking as PA.
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023005#comment-13023005 ]
Hudson commented on HDFS-1596:
------------------------------
Integrated in Hadoop-Hdfs-22-branch #35 (See [https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/])
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.21.1, 0.22.0, 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007638#comment-13007638 ]
Harsh J Chouraria commented on HDFS-1596:
-----------------------------------------
Great!
Could you also have a look at HADOOP-7117? Thank you! :)
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.21.1, 0.22.0, 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HDFS-1596:
-----------------------------------------
Resolution: Fixed
Fix Version/s: 0.22.0
0.21.1
Status: Resolved (was: Patch Available)
I have committed this. Thanks, Harsh!
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.21.1, 0.22.0, 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987121#action_12987121 ]
Harsh J Chouraria commented on HDFS-1596:
-----------------------------------------
Haven't these keys been deprecated, in favor of "dfs.namenode.checkpoint.*" prefixes, available in hdfs-default.xml? Is the move still required? Just clarifying before I attempt to do this.
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022495#comment-13022495 ]
Hudson commented on HDFS-1596:
------------------------------
Integrated in Hadoop-Hdfs-trunk #643 (See [https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/])
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.21.1, 0.22.0, 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007647#comment-13007647 ]
Hudson commented on HDFS-1596:
------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #562 (See [https://hudson.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/562/])
HDFS-1596. Replace fs.checkpoint.* with dfs.namenode.checkpoint.* in documentations. Contributed by Harsh J Chouraria
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.21.1, 0.22.0, 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Patrick Angeles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12987133#action_12987133 ]
Patrick Angeles commented on HDFS-1596:
---------------------------------------
I still see them in core-default.xml in trunk, so they should get removed or the deprecation should be documented in the xml file itself...
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated HDFS-1596:
------------------------------------
Attachment: HDFS-7117.r1.diff
Patch that updates all references of fs.checkpoint.* to their newer dfs.namenode.checkpoint.* keys.
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
> Attachments: HDFS-7117.r1.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007230#comment-13007230 ]
Hadoop QA commented on HDFS-1596:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12473714/HDFS-7117.r2.diff
against trunk revision 1081580.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.hdfs.TestFileAppend4
org.apache.hadoop.hdfs.TestLargeBlock
org.apache.hadoop.hdfs.TestWriteConfigurationToDFS
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/261//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/261//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/261//console
This message is automatically generated.
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated HDFS-1596:
------------------------------------
Status: Open (was: Patch Available)
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria reassigned HDFS-1596:
---------------------------------------
Assignee: Harsh J Chouraria
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Attachments: HDFS-7117.r1.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007602#comment-13007602 ]
Harsh J Chouraria commented on HDFS-1596:
-----------------------------------------
I have not a clue why these test cases started to fail with this patch. They do not seem related to the removal of these property refs (except from the area that marks it as deprecated).
Is it due to the HADOOP-7117 co-requirement [which involved changes to core-site.xml]?
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated HDFS-1596:
------------------------------------
Attachment: HDFS-7117.r2.diff
Ouch, looks like my IDE's re-arrangement of imports had struck again (Wasn't disabled then, is always disabled now).
Cleaned up the patch and re-upped. It can be applied cleanly now.
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.23.0
>
> Attachments: HDFS-7117.r1.diff, HDFS-7117.r2.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13000823#comment-13000823 ]
Hadoop QA commented on HDFS-1596:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12469584/HDFS-7117.r1.diff
against trunk revision 1075571.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/217//console
This message is automatically generated.
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.23.0
>
> Attachments: HDFS-7117.r1.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HDFS-1596) Move secondary namenode checkpoint
configs from core-default.xml to hdfs-default.xml
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-1596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated HDFS-1596:
------------------------------------
Tags: checkpoint, secondarynamenode
Fix Version/s: 0.23.0
Release Note: Removed references to the older fs.checkpoint.* properties that resided in core-site.xml
Status: Patch Available (was: Open)
Marking as Patch-Available.
> Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml
> ------------------------------------------------------------------------------------
>
> Key: HDFS-1596
> URL: https://issues.apache.org/jira/browse/HDFS-1596
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: Patrick Angeles
> Assignee: Harsh J Chouraria
> Fix For: 0.23.0
>
> Attachments: HDFS-7117.r1.diff
>
>
> The following configs are in core-default.xml, but are really read by the Secondary Namenode. These should be moved to hdfs-default.xml for consistency.
> <property>
> <name>fs.checkpoint.dir</name>
> <value>${hadoop.tmp.dir}/dfs/namesecondary</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary images to merge.
> If this is a comma-delimited list of directories then the image is
> replicated in all of the directories for redundancy.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.edits.dir</name>
> <value>${fs.checkpoint.dir}</value>
> <description>Determines where on the local filesystem the DFS secondary
> name node should store the temporary edits to merge.
> If this is a comma-delimited list of directoires then teh edits is
> replicated in all of the directoires for redundancy.
> Default value is same as fs.checkpoint.dir
> </description>
> </property>
> <property>
> <name>fs.checkpoint.period</name>
> <value>3600</value>
> <description>The number of seconds between two periodic checkpoints.
> </description>
> </property>
> <property>
> <name>fs.checkpoint.size</name>
> <value>67108864</value>
> <description>The size of the current edit log (in bytes) that triggers
> a periodic checkpoint even if the fs.checkpoint.period hasn't expired.
> </description>
> </property>
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira