You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Raghu Angadi (JIRA)" <ji...@apache.org> on 2009/06/12 03:30:07 UTC
[jira] Created: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
-----------------------------------------------------------------------------
Key: HADOOP-6017
URL: https://issues.apache.org/jira/browse/HADOOP-6017
Project: Hadoop Core
Issue Type: Bug
Affects Versions: 0.18.3
Reporter: Raghu Angadi
Assignee: Tsz Wo (Nicholas), SZE
Priority: Blocker
Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-6017:
-------------------------------------------
Attachment: 6017_20090611.patch
6017_20090611.patch: we should not use String.replaceFirst(..) at all.
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718705#action_12718705 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-6017:
------------------------------------------------
{noformat}
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 3 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
{noformat}
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720277#action_12720277 ]
Allen Wittenauer commented on HADOOP-6017:
------------------------------------------
Looking at the patch, this appears to only fix input/output validation. How do we deal with our currently corrupted edits file? This patch needs to fix that as well!
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch, HADOOP-6017-branch-20.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718940#action_12718940 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-6017:
------------------------------------------------
I ran unit tests on trunk with 6017_20090611b.patch. The only failed test was TestReduceFetch, which seemed unrelated. Then, I ran TestReduceFetch on a clean trunk. It also failed in both my linux and windows machines.
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718691#action_12718691 ]
Raghu Angadi commented on HADOOP-6017:
--------------------------------------
Stacktrace of such a failure:
{noformat}
2009-06-11 07:14:30,798 ERROR org.apache.hadoop.dfs.NameNode.Secondary:
Throwable Exception in doCheckpoint:
2009-06-11 07:14:30,798 ERROR org.apache.hadoop.dfs.NameNode.Secondary:
java.lang.IllegalArgumentException: Illegal group reference
at java.util.regex.Matcher.appendReplacement(Matcher.java:713)
at java.util.regex.Matcher.replaceFirst(Matcher.java:861)
at java.lang.String.replaceFirst(String.java:2147)
at
org.apache.hadoop.dfs.LeaseManager.changeLease(LeaseManager.java:288)
at
org.apache.hadoop.dfs.FSNamesystem.changeLease(FSNamesystem.java:4441)
at org.apache.hadoop.dfs.FSEditLog.loadFSEdits(FSEditLog.java:563)
at org.apache.hadoop.dfs.FSImage.loadFSEdits(FSImage.java:846)
at
org.apache.hadoop.dfs.SecondaryNameNode$CheckpointStorage.doMerge(SecondaryNameNode.java:567)
at
org.apache.hadoop.dfs.SecondaryNameNode$CheckpointStorage.access$000(SecondaryNameNode.java:464)
at
org.apache.hadoop.dfs.SecondaryNameNode.doMerge(SecondaryNameNode.java:341)
at
org.apache.hadoop.dfs.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:305)
at
org.apache.hadoop.dfs.SecondaryNameNode.run(SecondaryNameNode.java:216)
at java.lang.Thread.run(Thread.java:619)
2009-06-11 07:14:30,842 INFO org.apache.hadoop.dfs.NameNode.Secondary:
SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down SecondaryNameNode at host/ip
************************************************************/
{noformat}
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-6017:
-------------------------------------------
Attachment: 6017_20090611b.patch
6017_20090611b.patch: changed the unit test a little bit.
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718952#action_12718952 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-6017:
------------------------------------------------
I filed HADOOP-6029 for TestReduceFetch.
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720316#action_12720316 ]
Raghu Angadi commented on HADOOP-6017:
--------------------------------------
This fixes actual bug. The edits file is not corrupted. It is just that NameNode didn't handle filenames properly for certain edit log entries. With this patch, NN and secondary NN can handle the same edit log properly.
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch, HADOOP-6017-branch-20.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-6017:
---------------------------------
Status: Patch Available (was: Open)
I am planning to commit these. Hudson is currently running Jun 11th patches.
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-6017:
-------------------------------------------
Component/s: dfs
Description:
SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
was:
SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
Hadoop Flags: [Reviewed]
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720844#action_12720844 ]
Hudson commented on HADOOP-6017:
--------------------------------
Integrated in Hadoop-trunk #869 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/869/])
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch, HADOOP-6017-branch-20.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-6017:
---------------------------------
Resolution: Fixed
Release Note: rename to a file name with certain characters can cause fatal exceptions in NameNode initialization and Secondary NameNode.
Status: Resolved (was: Patch Available)
I just committed this. Thanks Nicholas.
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-6017:
---------------------------------
Attachment: HADOOP-6017-branch-20.patch
Patch for branch 0.20 is attached. The trunk patch does not apply because of path difference for hdfs tests.
This patch applies to 0.19 as well.
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch, HADOOP-6017-branch-20.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6017) NameNode and SecondaryNameNode fail
to restart because of abnormal filenames.
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raghu Angadi updated HADOOP-6017:
---------------------------------
Attachment: HADOOP-6017-branch-18.patch
+1. The patch looks good.
I am attaching the patch 0.18 branch.
There are more changes required for TestRenameWhileOpen.java since the rename test that the new test is part of was disabled in 0.18. This patch enables parts of the test.
> NameNode and SecondaryNameNode fail to restart because of abnormal filenames.
> -----------------------------------------------------------------------------
>
> Key: HADOOP-6017
> URL: https://issues.apache.org/jira/browse/HADOOP-6017
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.18.3
> Reporter: Raghu Angadi
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.4, 0.19.2, 0.20.1, 0.21.0
>
> Attachments: 6017_20090611.patch, 6017_20090611b.patch, HADOOP-6017-branch-18.patch
>
>
> SecondaryNameNode (and NameNode) fail to load the edits log. I will include stack trace in next comment.
> This is traced to the fact that LeaseManager uses {{String.relaceFirst()}} to replace front of a sting with another string. Unfortunately {{replaceFirst()}} uses regex, though the first argument is {{quoted}} by the code, the second argument is not. (the second arg is not really treated as regex but still gets processed for back references (as in '{{sed s/first/second/g}}')
> As Nicholas suggested, it is just simpler to use {{substring()}} to replace part of the string.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.