You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2007/04/14 00:01:15 UTC
[jira] Created: (HADOOP-1261) Restart of the same data-node should
not generate edits log records.
Restart of the same data-node should not generate edits log records.
--------------------------------------------------------------------
Key: HADOOP-1261
URL: https://issues.apache.org/jira/browse/HADOOP-1261
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.12.0
Reporter: Konstantin Shvachko
Assigned To: Raghu Angadi
Priority: Minor
Fix For: 0.13.0
Currently during registration of a data-node with the storage id that has already been registered
the name-node logs 2 records: <add node1> <remove node2>.
If the same node has been restarted these two records are redundant, lead to unnecessary
increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1261) Restart of the same data-node
should not generate edits log records.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489213 ]
Konstantin Shvachko commented on HADOOP-1261:
---------------------------------------------
Could you check whether we can also get rid of clusterMap.remove() and clusterMap.add() in case nodeN == nodeS.
It seems logical not to do anything if the same node is restarting.
Also I see dnReg is used only to set a correct ip address in nodeReg. There should be an easier way.
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assigned To: Raghu Angadi
> Priority: Minor
> Fix For: 0.13.0
>
> Attachments: HADOOP-1261-1.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1261) Restart of the same data-node should
not generate edits log records.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-1261:
---------------------------------
Attachment: HADOOP-1261-1.patch
We avoid writing the log entry when "nodeS == nodeN" (i.e. node existed before and was serving the same storage id ).
Konstantin, could you take a quick look at the patch?
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assigned To: Raghu Angadi
> Priority: Minor
> Fix For: 0.13.0
>
> Attachments: HADOOP-1261-1.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1261) Restart of the same data-node should
not generate edits log records.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-1261:
---------------------------------
Status: Patch Available (was: Open)
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: HADOOP-1261-1.patch, HADOOP-1261-2.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1261) Restart of the same data-node should
not generate edits log records.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-1261:
---------------------------------
Attachment: HADOOP-1261-2.patch
Updated for for trunk.
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: HADOOP-1261-1.patch, HADOOP-1261-2.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1261) Restart of the same data-node
should not generate edits log records.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500269 ]
Hadoop QA commented on HADOOP-1261:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12358585/HADOOP-1261-2.patch applied and successfully tested against trunk revision r542595.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/220/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/220/console
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: HADOOP-1261-1.patch, HADOOP-1261-2.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1261) Restart of the same data-node should
not generate edits log records.
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1261:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Raghu!
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: HADOOP-1261-1.patch, HADOOP-1261-2.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1261) Restart of the same data-node
should not generate edits log records.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12500240 ]
Hadoop QA commented on HADOOP-1261:
-----------------------------------
-1, could not apply patch.
The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12355633/HADOOP-1261-1.patch as a patch to trunk revision r542595.
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/219/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: HADOOP-1261-1.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1261) Restart of the same data-node should
not generate edits log records.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-1261:
---------------------------------
Status: Open (was: Patch Available)
need to regenerate
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: HADOOP-1261-1.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1261) Restart of the same data-node
should not generate edits log records.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489215 ]
Raghu Angadi commented on HADOOP-1261:
--------------------------------------
> Could you check whether we can also get rid of clusterMap.remove() and clusterMap.add() in case nodeN == nodeS.
Then we also need to make sure that the node is on the same rack etc. I am not sure if the extra code is worth it. Node registration is not supposed to be very often. Simpler it is, the better.
> Also I see dnReg is used only to set a correct ip address in nodeReg. There should be an easier way.
hm. will check. Currently it just uses existing interface.
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assigned To: Raghu Angadi
> Priority: Minor
> Fix For: 0.13.0
>
> Attachments: HADOOP-1261-1.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1261) Restart of the same data-node should
not generate edits log records.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-1261:
---------------------------------
Fix Version/s: 0.14.0
Status: Patch Available (was: Open)
getting rid of dnReg could be another minor clean up Jira.
> Restart of the same data-node should not generate edits log records.
> --------------------------------------------------------------------
>
> Key: HADOOP-1261
> URL: https://issues.apache.org/jira/browse/HADOOP-1261
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.12.0
> Reporter: Konstantin Shvachko
> Assignee: Raghu Angadi
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: HADOOP-1261-1.patch
>
>
> Currently during registration of a data-node with the storage id that has already been registered
> the name-node logs 2 records: <add node1> <remove node2>.
> If the same node has been restarted these two records are redundant, lead to unnecessary
> increase of the edits file and result in a slow-down of the name-node startup.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.