You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2007/03/31 00:53:25 UTC
[jira] Created: (HADOOP-1188) processIOError() should update fstime
file
processIOError() should update fstime file
------------------------------------------
Key: HADOOP-1188
URL: https://issues.apache.org/jira/browse/HADOOP-1188
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.12.2, 0.11.2, 0.10.1
Reporter: Konstantin Shvachko
Fix For: 0.13.0
Name-node can have multiple directories to store the name space image and edits.
During startup the name-node selects the latest image and reads it in memory.
fstime stores the time of the latest checkpoint.
If one of the directories becomes inaccessible during normal operation the name-node
excludes it from the list, and never writes anything into it until restarted.
Now if the the name-node restarts after excluding one of the directories, and if that
excluded directory contained the latest image, then all modifications of the name space
recorded in edits files in other directories starting from the moment the directory was
excluded will be lost.
We should update fstime in all remaining good directories after the exclusion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1188) processIOError() should update fstime
file
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1188:
-------------------------------------
Fix Version/s: 0.15.0
> processIOError() should update fstime file
> ------------------------------------------
>
> Key: HADOOP-1188
> URL: https://issues.apache.org/jira/browse/HADOOP-1188
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.10.1, 0.11.2, 0.12.2
> Reporter: Konstantin Shvachko
> Fix For: 0.15.0
>
>
> Name-node can have multiple directories to store the name space image and edits.
> During startup the name-node selects the latest image and reads it in memory.
> fstime stores the time of the latest checkpoint.
> If one of the directories becomes inaccessible during normal operation the name-node
> excludes it from the list, and never writes anything into it until restarted.
> Now if the the name-node restarts after excluding one of the directories, and if that
> excluded directory contained the latest image, then all modifications of the name space
> recorded in edits files in other directories starting from the moment the directory was
> excluded will be lost.
> We should update fstime in all remaining good directories after the exclusion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1188) processIOError() should update
fstime file
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12486055 ]
dhruba borthakur commented on HADOOP-1188:
------------------------------------------
+1. Sounds great.
> processIOError() should update fstime file
> ------------------------------------------
>
> Key: HADOOP-1188
> URL: https://issues.apache.org/jira/browse/HADOOP-1188
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.10.1, 0.11.2, 0.12.2
> Reporter: Konstantin Shvachko
> Fix For: 0.13.0
>
>
> Name-node can have multiple directories to store the name space image and edits.
> During startup the name-node selects the latest image and reads it in memory.
> fstime stores the time of the latest checkpoint.
> If one of the directories becomes inaccessible during normal operation the name-node
> excludes it from the list, and never writes anything into it until restarted.
> Now if the the name-node restarts after excluding one of the directories, and if that
> excluded directory contained the latest image, then all modifications of the name space
> recorded in edits files in other directories starting from the moment the directory was
> excluded will be lost.
> We should update fstime in all remaining good directories after the exclusion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1188) processIOError() should update
fstime file
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12553528 ]
dhruba borthakur commented on HADOOP-1188:
------------------------------------------
It is possible that updating the fstime on the good directories could fail too. So, a namenode restart should pick all those directories that have the latest (and same) fstime and fail to restart if the sizes of the edit files in those directories are not exactly the same.
> processIOError() should update fstime file
> ------------------------------------------
>
> Key: HADOOP-1188
> URL: https://issues.apache.org/jira/browse/HADOOP-1188
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.10.1, 0.11.2, 0.12.2
> Reporter: Konstantin Shvachko
>
> Name-node can have multiple directories to store the name space image and edits.
> During startup the name-node selects the latest image and reads it in memory.
> fstime stores the time of the latest checkpoint.
> If one of the directories becomes inaccessible during normal operation the name-node
> excludes it from the list, and never writes anything into it until restarted.
> Now if the the name-node restarts after excluding one of the directories, and if that
> excluded directory contained the latest image, then all modifications of the name space
> recorded in edits files in other directories starting from the moment the directory was
> excluded will be lost.
> We should update fstime in all remaining good directories after the exclusion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1188) processIOError() should update
fstime file
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur reassigned HADOOP-1188:
----------------------------------------
Assignee: Konstantin Shvachko
> processIOError() should update fstime file
> ------------------------------------------
>
> Key: HADOOP-1188
> URL: https://issues.apache.org/jira/browse/HADOOP-1188
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.10.1, 0.11.2, 0.12.2
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.16.0
>
>
> Name-node can have multiple directories to store the name space image and edits.
> During startup the name-node selects the latest image and reads it in memory.
> fstime stores the time of the latest checkpoint.
> If one of the directories becomes inaccessible during normal operation the name-node
> excludes it from the list, and never writes anything into it until restarted.
> Now if the the name-node restarts after excluding one of the directories, and if that
> excluded directory contained the latest image, then all modifications of the name space
> recorded in edits files in other directories starting from the moment the directory was
> excluded will be lost.
> We should update fstime in all remaining good directories after the exclusion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1188) processIOError() should update fstime
file
Posted by "Robert Chansler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Chansler updated HADOOP-1188:
------------------------------------
Priority: Blocker (was: Major)
Promoted for consideration in 16.
> processIOError() should update fstime file
> ------------------------------------------
>
> Key: HADOOP-1188
> URL: https://issues.apache.org/jira/browse/HADOOP-1188
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.10.1, 0.11.2, 0.12.2
> Reporter: Konstantin Shvachko
> Priority: Blocker
>
> Name-node can have multiple directories to store the name space image and edits.
> During startup the name-node selects the latest image and reads it in memory.
> fstime stores the time of the latest checkpoint.
> If one of the directories becomes inaccessible during normal operation the name-node
> excludes it from the list, and never writes anything into it until restarted.
> Now if the the name-node restarts after excluding one of the directories, and if that
> excluded directory contained the latest image, then all modifications of the name space
> recorded in edits files in other directories starting from the moment the directory was
> excluded will be lost.
> We should update fstime in all remaining good directories after the exclusion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1188) processIOError() should update fstime
file
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-1188:
--------------------------------
Fix Version/s: (was: 0.16.0)
0.16.1
> processIOError() should update fstime file
> ------------------------------------------
>
> Key: HADOOP-1188
> URL: https://issues.apache.org/jira/browse/HADOOP-1188
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.10.1, 0.11.2, 0.12.2
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.16.1
>
>
> Name-node can have multiple directories to store the name space image and edits.
> During startup the name-node selects the latest image and reads it in memory.
> fstime stores the time of the latest checkpoint.
> If one of the directories becomes inaccessible during normal operation the name-node
> excludes it from the list, and never writes anything into it until restarted.
> Now if the the name-node restarts after excluding one of the directories, and if that
> excluded directory contained the latest image, then all modifications of the name space
> recorded in edits files in other directories starting from the moment the directory was
> excluded will be lost.
> We should update fstime in all remaining good directories after the exclusion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1188) processIOError() should update fstime
file
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HADOOP-1188:
-------------------------------------
Fix Version/s: 0.16.0
> processIOError() should update fstime file
> ------------------------------------------
>
> Key: HADOOP-1188
> URL: https://issues.apache.org/jira/browse/HADOOP-1188
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.10.1, 0.11.2, 0.12.2
> Reporter: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.16.0
>
>
> Name-node can have multiple directories to store the name space image and edits.
> During startup the name-node selects the latest image and reads it in memory.
> fstime stores the time of the latest checkpoint.
> If one of the directories becomes inaccessible during normal operation the name-node
> excludes it from the list, and never writes anything into it until restarted.
> Now if the the name-node restarts after excluding one of the directories, and if that
> excluded directory contained the latest image, then all modifications of the name space
> recorded in edits files in other directories starting from the moment the directory was
> excluded will be lost.
> We should update fstime in all remaining good directories after the exclusion.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.