You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2008/12/30 20:07:44 UTC
[jira] Created: (HADOOP-4957) Lease monitor should not fail on a
live name-node.
Lease monitor should not fail on a live name-node.
--------------------------------------------------
Key: HADOOP-4957
URL: https://issues.apache.org/jira/browse/HADOOP-4957
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.18.3
Reporter: Konstantin Shvachko
Assignee: Tsz Wo (Nicholas), SZE
Fix For: 0.20.0
If a runtime exception is thrown inside the lease monitor, as it happened in HADOOP-4951, the monitor will die although the name-node will continue to run. The problem is that abandoned files will not be garbage collected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4957) Lease monitor should not fail on a
live name-node.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659922#action_12659922 ]
Konstantin Shvachko commented on HADOOP-4957:
---------------------------------------------
> Are you saying that we should let the monitor run even there is a bug in it?
I am saying that this is inconsistent: either the name-node should die with the monitor or the monitor should continue to run if the name-node is running.
> Lease monitor should not fail on a live name-node.
> --------------------------------------------------
>
> Key: HADOOP-4957
> URL: https://issues.apache.org/jira/browse/HADOOP-4957
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.0
>
>
> If a runtime exception is thrown inside the lease monitor, as it happened in HADOOP-4951, the monitor will die although the name-node will continue to run. The problem is that abandoned files will not be garbage collected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4957) Lease monitor should not fail on a
live name-node.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659902#action_12659902 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4957:
------------------------------------------------
>If a runtime exception is thrown inside the lease monitor, as it happened in HADOOP-4951, the monitor will die although the name-node will continue to run.
But HADOOP-4951 happened because there was a bug in the code. Are you saying that we should let the monitor run even there is a bug in it?
> we should check other name-node monitors to do the same thing; better create a base class for monitors implementing this logic.
Took a quick look, FSNamesystem.SafeModeMonitor is not doing the thing you described.
> Lease monitor should not fail on a live name-node.
> --------------------------------------------------
>
> Key: HADOOP-4957
> URL: https://issues.apache.org/jira/browse/HADOOP-4957
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.0
>
>
> If a runtime exception is thrown inside the lease monitor, as it happened in HADOOP-4951, the monitor will die although the name-node will continue to run. The problem is that abandoned files will not be garbage collected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4957) Lease monitor should not fail on a
live name-node.
Posted by "Lee Tucker (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12685399#action_12685399 ]
Lee Tucker commented on HADOOP-4957:
------------------------------------
I'll be responding to mail on 3/24.
> Lease monitor should not fail on a live name-node.
> --------------------------------------------------
>
> Key: HADOOP-4957
> URL: https://issues.apache.org/jira/browse/HADOOP-4957
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Konstantin Shvachko
>
> If a runtime exception is thrown inside the lease monitor, as it happened in HADOOP-4951, the monitor will die although the name-node will continue to run. The problem is that abandoned files will not be garbage collected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4957) Lease monitor should not fail on a
live name-node.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660135#action_12660135 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4957:
------------------------------------------------
> 1. lease monitor should catch all exceptions log an error and continue to run until name-node is running;
> 2. lease monitor should loop until the name-node is FSNamesystem.fsRunning;
If we do the above, it will easily end up an infinite loop when there is a runtime exception. Consider the case that the oldest lease have a problem before it gets renew or remove from the queue. Then it will keep sitting on the top of the queue. The monitor will keep checking the same lease, catching the same runtime exception again and again.
> Lease monitor should not fail on a live name-node.
> --------------------------------------------------
>
> Key: HADOOP-4957
> URL: https://issues.apache.org/jira/browse/HADOOP-4957
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.0
>
>
> If a runtime exception is thrown inside the lease monitor, as it happened in HADOOP-4951, the monitor will die although the name-node will continue to run. The problem is that abandoned files will not be garbage collected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4957) Lease monitor should not fail on a
live name-node.
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660146#action_12660146 ]
Nigel Daley commented on HADOOP-4957:
-------------------------------------
A reminder that our Code Review Guidelines (http://wiki.apache.org/hadoop/CodeReviewChecklist) include:
- look for accidental propagation of exceptions
- look for unanticipated runtime exceptions
If these guidelines were more consistently followed, they types of problems could be averted.
> Lease monitor should not fail on a live name-node.
> --------------------------------------------------
>
> Key: HADOOP-4957
> URL: https://issues.apache.org/jira/browse/HADOOP-4957
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.0
>
>
> If a runtime exception is thrown inside the lease monitor, as it happened in HADOOP-4951, the monitor will die although the name-node will continue to run. The problem is that abandoned files will not be garbage collected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4957) Lease monitor should not fail on a
live name-node.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659896#action_12659896 ]
Konstantin Shvachko commented on HADOOP-4957:
---------------------------------------------
I propose:
# lease monitor should catch all exceptions log an error and continue to run until name-node is running;
# lease monitor should loop until the name-node is {{FSNamesystem.fsRunning}};
# we should check other name-node monitors to do the same thing; better create a base class for monitors implementing this logic.
# we should also take care of the of converting FileUnderConstruction into real files as stated in
https://issues.apache.org/jira/browse/HADOOP-4795?focusedCommentId=12655006#action_12655006
> Lease monitor should not fail on a live name-node.
> --------------------------------------------------
>
> Key: HADOOP-4957
> URL: https://issues.apache.org/jira/browse/HADOOP-4957
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Konstantin Shvachko
> Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.0
>
>
> If a runtime exception is thrown inside the lease monitor, as it happened in HADOOP-4951, the monitor will die although the name-node will continue to run. The problem is that abandoned files will not be garbage collected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-4957) Lease monitor should not fail on a
live name-node.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE resolved HADOOP-4957.
--------------------------------------------
Resolution: Won't Fix
Fix Version/s: (was: 0.20.0)
Assignee: (was: Tsz Wo (Nicholas), SZE)
> Lease monitor should not fail on a live name-node.
> --------------------------------------------------
>
> Key: HADOOP-4957
> URL: https://issues.apache.org/jira/browse/HADOOP-4957
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.18.3
> Reporter: Konstantin Shvachko
>
> If a runtime exception is thrown inside the lease monitor, as it happened in HADOOP-4951, the monitor will die although the name-node will continue to run. The problem is that abandoned files will not be garbage collected.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.