You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2008/12/06 01:25:44 UTC
[jira] Created: (HADOOP-4795) Lease monitor may get into an
infinite loop
Lease monitor may get into an infinite loop
-------------------------------------------
Key: HADOOP-4795
URL: https://issues.apache.org/jira/browse/HADOOP-4795
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Reporter: Tsz Wo (Nicholas), SZE
Priority: Blocker
Fix For: 0.18.3
If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4795:
-------------------------------------------
Attachment: 4795_20081209.patch
4795_20081209.patch: incorporated Konstantin's comments.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655006#action_12655006 ]
Konstantin Shvachko commented on HADOOP-4795:
---------------------------------------------
+1.
If the code in internalReleaseLease() below the new IOExceptions you introduced would throw an exception the data-structures may become inconsistent. But as I can see these exceptions are not thrown anywhere below under the conditions. We should commit this.
Another concern that existing images will contain FileUnderConstruction, which will unclosable forever, and therefore we should investigate how to convert them into real files in a separate jira.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch, 4795_20081209_0.18.patch, TestLeaseMonitor.java
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655389#action_12655389 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4795:
------------------------------------------------
{noformat}
[exec] -1 overall.
[exec] +1 @author. The patch does not contain any @author tags.
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
{noformat}
Tested manually. No new tests added.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch, 4795_20081209_0.18.patch, TestLeaseMonitor.java
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4795:
-------------------------------------------
Attachment: 4795_20081208.patch
> In the method check(), if the older entry has not expired, we should break out of the while loop?
You are right. I removed the "break;" in the original codes but forgot to add a "return;"
4795_20081208.patch: fix the bug above. Will try to add a test.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4795:
-------------------------------------------
Attachment: 4795_20081209_0.18.patch
4795_20081209_0.18.patch: for 0.18
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch, 4795_20081209_0.18.patch
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4795:
-------------------------------------------
Assignee: Tsz Wo (Nicholas), SZE
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch, 4795_20081209_0.18.patch, TestLeaseMonitor.java
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655650#action_12655650 ]
Hudson commented on HADOOP-4795:
--------------------------------
Integrated in Hadoop-trunk #685 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/685/])
. Prevent lease monitor getting into an infinite loop when leases and the namespace tree does not match. (szetszwo)
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch, 4795_20081209_0.18.patch, 4795_20081209_0.19.patch, TestLeaseMonitor.java
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4795:
-------------------------------------------
Attachment: TestLeaseMonitor.java
TestLeaseMonitor.java: This is a manual test for the infinite loop problem.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch, 4795_20081209_0.18.patch, TestLeaseMonitor.java
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654308#action_12654308 ]
Suresh Srinivas commented on HADOOP-4795:
-----------------------------------------
In the method {{check()}}, if the older entry has not expired, we should break out of the while loop? Otherwise this code will cause a tight loop until {{sortedLease.size()}} becomes 0 in size?
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4795:
-------------------------------------------
Attachment: 4795_20081205.patch
4795_20081205.patch: remove the lease if the corresponding INode is not found.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4795:
-------------------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch, 4795_20081209_0.18.patch, 4795_20081209_0.19.patch, TestLeaseMonitor.java
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655444#action_12655444 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4795:
------------------------------------------------
Tested locally, only org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.TestStartAtOffset failed. It does not seem related to this patch. Will file a issue I see it failing again.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch, 4795_20081209_0.18.patch, TestLeaseMonitor.java
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654004#action_12654004 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4795:
------------------------------------------------
> lease is not found in the namespace for some reasons (e.g. bugs)
Let me clarify this: the file creation lease is in the system but the corresponding file is not found. This should not happen normally.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-4795:
-------------------------------------------
Attachment: 4795_20081209_0.19.patch
4795_20081209_0.19.patch: for 0.19
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch, 4795_20081209.patch, 4795_20081209_0.18.patch, 4795_20081209_0.19.patch, TestLeaseMonitor.java
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4795) Lease monitor may get into an
infinite loop
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654608#action_12654608 ]
Konstantin Shvachko commented on HADOOP-4795:
---------------------------------------------
# I would rename Monitor.check() to something like checkLeases().
# {{esle}} statement can be removed.
# Wrapping String Collection into {{ArrayList}} can be replaced by collecting paths to be removed in an {{ArrayList}} and then removing them afterwards in a separate loop. This will be much less copying.
> Lease monitor may get into an infinite loop
> -------------------------------------------
>
> Key: HADOOP-4795
> URL: https://issues.apache.org/jira/browse/HADOOP-4795
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Fix For: 0.18.3
>
> Attachments: 4795_20081205.patch, 4795_20081208.patch
>
>
> If a lease is not found in the namespace for some reasons (e.g. bugs), lease monitor may get into an infinite loop.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.