You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Colin Patrick McCabe (JIRA)" <ji...@apache.org> on 2012/04/21 02:59:33 UTC
[jira] [Created] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Colin Patrick McCabe created HDFS-3310:
------------------------------------------
Summary: Make sure that we abort when no edit log directories are left
Key: HDFS-3310
URL: https://issues.apache.org/jira/browse/HDFS-3310
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 0.20.1
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HDFS-3310:
---------------------------------------
Attachment: HDFS-3310-b1.002.patch
* address Nicholas' comment (use exitIfNoStreams more consistently)
* add new unit test in TestStorageDirectoryFailure
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch, HDFS-3310-b1.002.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262254#comment-13262254 ]
Eli Collins commented on HDFS-3310:
-----------------------------------
Mind updating the calls to updateRemovedDirs(sd, null) to use your new method w/o 2nd param? No need to re-run test-patch for this.
Otherwise +1, looks great. Nice find btw!
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch, HDFS-3310-b1.002.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-3310:
------------------------------
Target Version/s: 1.1.0, 1.0.3 (was: 0.20.1)
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HDFS-3310:
---------------------------------------
Attachment: HDFS-3310-b1.001.patch
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262783#comment-13262783 ]
Colin Patrick McCabe commented on HDFS-3310:
--------------------------------------------
I re-ran all unit tests with this change and saw no more than the usual failures (TestCLI, TestFileAppend4, etc.).
I tried to run the test-patch script, but there seem to be a lot of environment setup issues at the moment.
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch, HDFS-3310-b1.002.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262981#comment-13262981 ]
Colin Patrick McCabe commented on HDFS-3310:
--------------------------------------------
I ran testpatch (sorry for the delay) and got the expected 8 warnings. (see HADOOP-7847). So it looks good.
C.
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch, HDFS-3310-b1.002.patch, HDFS-3310-b1.003.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins resolved HDFS-3310.
-------------------------------
Resolution: Fixed
Fix Version/s: 1.0.3
Target Version/s: (was: 1.0.3, 1.1.0)
Hadoop Flags: Reviewed
I've committed this to branch-1 and merged for 1.0.3.
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Fix For: 1.0.3
>
> Attachments: HDFS-3310-b1.001.patch, HDFS-3310-b1.002.patch, HDFS-3310-b1.003.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263085#comment-13263085 ]
Eli Collins commented on HDFS-3310:
-----------------------------------
+1 Nice work Colin!
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch, HDFS-3310-b1.002.patch, HDFS-3310-b1.003.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HDFS-3310:
---------------------------------------
Attachment: HDFS-3310-b1.003.patch
* use the one-argument form of FsImage#updateRemovedDirs where appropriate
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch, HDFS-3310-b1.002.patch, HDFS-3310-b1.003.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261205#comment-13261205 ]
Tsz Wo (Nicholas), SZE commented on HDFS-3310:
----------------------------------------------
+1 the new patch looks good.
Could you run "test-patch" and all unit tests?
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch, HDFS-3310-b1.002.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3310) Make sure that we abort when no edit
log directories are left
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261051#comment-13261051 ]
Tsz Wo (Nicholas), SZE commented on HDFS-3310:
----------------------------------------------
I think should be replaced by exitIfNoStreams().
{code}
+ if (editStreams.isEmpty()) {
+ throw new IOException("no editlog streams");
+ }
{code}
Patch looks good other then that.
> Make sure that we abort when no edit log directories are left
> -------------------------------------------------------------
>
> Key: HDFS-3310
> URL: https://issues.apache.org/jira/browse/HDFS-3310
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.20.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-3310-b1.001.patch
>
>
> We should make sure to abort when there are no edit log directories left to write to. It seems that there is at least one case that is slipping through the cracks right now in branch-1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira