You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2009/01/10 00:57:59 UTC
[jira] Created: (HADOOP-5009) DataNode#shutdown sometimes leaves
data block scanner verification log unclosed
DataNode#shutdown sometimes leaves data block scanner verification log unclosed
-------------------------------------------------------------------------------
Key: HADOOP-5009
URL: https://issues.apache.org/jira/browse/HADOOP-5009
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.17.0
Reporter: Hairong Kuang
Assignee: Hairong Kuang
Fix For: 0.19.1
When datanode gets shutdown by calling DataNode#shutdown, it occasionally leaves the data block scanner verification log unclosed. There are two possible causes:
1. DataNode does not wait until block scanner thread to exit.
2. DataBlockScanner does not guarantee that the verification log is closed if the scanner is interrupted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5009) DataNode#shutdown sometimes leaves
data block scanner verification log unclosed
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12665976#action_12665976 ]
Hairong Kuang commented on HADOOP-5009:
---------------------------------------
[exec] -1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] -1 tests included. The patch doesn't appear to include any new or modified tests.
[exec] Please justify why no tests are needed for this patch.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
[exec]
[exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
ant test-core fails only TestMapReduceLocal which is a known issue.
> DataNode#shutdown sometimes leaves data block scanner verification log unclosed
> -------------------------------------------------------------------------------
>
> Key: HADOOP-5009
> URL: https://issues.apache.org/jira/browse/HADOOP-5009
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.19.1
>
> Attachments: blockScannerFd.patch, blockScannerFd1.patch
>
>
> When datanode gets shutdown by calling DataNode#shutdown, it occasionally leaves the data block scanner verification log unclosed. There are two possible causes:
> 1. DataNode does not wait until block scanner thread to exit.
> 2. DataBlockScanner does not guarantee that the verification log is closed if the scanner is interrupted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5009) DataNode#shutdown sometimes leaves
data block scanner verification log unclosed
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-5009:
----------------------------------
Attachment: blockScannerFd.patch
The attached patch attempts to fix the problem.
> DataNode#shutdown sometimes leaves data block scanner verification log unclosed
> -------------------------------------------------------------------------------
>
> Key: HADOOP-5009
> URL: https://issues.apache.org/jira/browse/HADOOP-5009
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.19.1
>
> Attachments: blockScannerFd.patch
>
>
> When datanode gets shutdown by calling DataNode#shutdown, it occasionally leaves the data block scanner verification log unclosed. There are two possible causes:
> 1. DataNode does not wait until block scanner thread to exit.
> 2. DataBlockScanner does not guarantee that the verification log is closed if the scanner is interrupted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5009) DataNode#shutdown sometimes leaves
data block scanner verification log unclosed
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-5009:
----------------------------------
Status: Patch Available (was: Open)
> DataNode#shutdown sometimes leaves data block scanner verification log unclosed
> -------------------------------------------------------------------------------
>
> Key: HADOOP-5009
> URL: https://issues.apache.org/jira/browse/HADOOP-5009
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.19.1
>
> Attachments: blockScannerFd.patch, blockScannerFd1.patch
>
>
> When datanode gets shutdown by calling DataNode#shutdown, it occasionally leaves the data block scanner verification log unclosed. There are two possible causes:
> 1. DataNode does not wait until block scanner thread to exit.
> 2. DataBlockScanner does not guarantee that the verification log is closed if the scanner is interrupted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5009) DataNode#shutdown sometimes leaves
data block scanner verification log unclosed
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-5009:
----------------------------------
Attachment: blockScannerFd1.patch
This patch incorporates Nicholas' comment.
> DataNode#shutdown sometimes leaves data block scanner verification log unclosed
> -------------------------------------------------------------------------------
>
> Key: HADOOP-5009
> URL: https://issues.apache.org/jira/browse/HADOOP-5009
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.19.1
>
> Attachments: blockScannerFd.patch, blockScannerFd1.patch
>
>
> When datanode gets shutdown by calling DataNode#shutdown, it occasionally leaves the data block scanner verification log unclosed. There are two possible causes:
> 1. DataNode does not wait until block scanner thread to exit.
> 2. DataBlockScanner does not guarantee that the verification log is closed if the scanner is interrupted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5009) DataNode#shutdown sometimes leaves
data block scanner verification log unclosed
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12663129#action_12663129 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-5009:
------------------------------------------------
+1 Patch looks good. Two minor suggestions:
- using blockScannerThread.join() may be dangerous if blockScannerThread does not terminate. How about replace join() with blockScannerThread.join(millis), so that it will time out?
- I think it is better to use try-finally in closing the readers. See [a comment|https://issues.apache.org/jira/browse/HADOOP-2926?focusedCommentId=12574664#action_12574664] in HADOOP-2926.
> DataNode#shutdown sometimes leaves data block scanner verification log unclosed
> -------------------------------------------------------------------------------
>
> Key: HADOOP-5009
> URL: https://issues.apache.org/jira/browse/HADOOP-5009
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.19.1
>
> Attachments: blockScannerFd.patch
>
>
> When datanode gets shutdown by calling DataNode#shutdown, it occasionally leaves the data block scanner verification log unclosed. There are two possible causes:
> 1. DataNode does not wait until block scanner thread to exit.
> 2. DataBlockScanner does not guarantee that the verification log is closed if the scanner is interrupted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5009) DataNode#shutdown sometimes leaves
data block scanner verification log unclosed
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-5009:
-------------------------------------------
Hadoop Flags: [Reviewed]
+1 the new patch looks good.
> DataNode#shutdown sometimes leaves data block scanner verification log unclosed
> -------------------------------------------------------------------------------
>
> Key: HADOOP-5009
> URL: https://issues.apache.org/jira/browse/HADOOP-5009
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.19.1
>
> Attachments: blockScannerFd.patch, blockScannerFd1.patch
>
>
> When datanode gets shutdown by calling DataNode#shutdown, it occasionally leaves the data block scanner verification log unclosed. There are two possible causes:
> 1. DataNode does not wait until block scanner thread to exit.
> 2. DataBlockScanner does not guarantee that the verification log is closed if the scanner is interrupted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5009) DataNode#shutdown sometimes leaves
data block scanner verification log unclosed
Posted by "Hairong Kuang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HADOOP-5009:
----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I've just resolved this.
> DataNode#shutdown sometimes leaves data block scanner verification log unclosed
> -------------------------------------------------------------------------------
>
> Key: HADOOP-5009
> URL: https://issues.apache.org/jira/browse/HADOOP-5009
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.19.1
>
> Attachments: blockScannerFd.patch, blockScannerFd1.patch
>
>
> When datanode gets shutdown by calling DataNode#shutdown, it occasionally leaves the data block scanner verification log unclosed. There are two possible causes:
> 1. DataNode does not wait until block scanner thread to exit.
> 2. DataBlockScanner does not guarantee that the verification log is closed if the scanner is interrupted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.