You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jimmy Xiang (JIRA)" <ji...@apache.org> on 2012/10/11 21:01:03 UTC
[jira] [Created] (HBASE-6979) recovered.edits file should no break
SplitLogManager
Jimmy Xiang created HBASE-6979:
----------------------------------
Summary: recovered.edits file should no break SplitLogManager
Key: HBASE-6979
URL: https://issues.apache.org/jira/browse/HBASE-6979
Project: HBase
Issue Type: Improvement
Components: master
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6979) recovered.edits file should not
break distributed log splitting
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476812#comment-13476812 ]
stack commented on HBASE-6979:
------------------------------
I'm ok w/ this. /tmp is better than no where. And this is a mess when folks run into it -- if they ever do. This is good clean up in that case.
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6979) recovered.edits file should no break
SplitLogManager
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6979:
-------------------------------
Status: Patch Available (was: Open)
In this patch, we just sideline any existing old recovered.edits file. No special care is tried to make sure the old file is indeed sidelined properly, don't overwrite other file.
> recovered.edits file should no break SplitLogManager
> ----------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6979) recovered.edits file should not break
distributed log splitting
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6979:
-------------------------------
Resolution: Fixed
Fix Version/s: 0.96.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Stack for the review. Integrated into trunk.
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6979) recovered.edits file should not
break distributed log splitting
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474634#comment-13474634 ]
Hadoop QA commented on HBASE-6979:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548816/trunk-6979.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3036//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3036//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3036//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3036//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3036//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3036//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3036//console
This message is automatically generated.
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6979) recovered.edits file should not
break distributed log splitting
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476280#comment-13476280 ]
Jimmy Xiang commented on HBASE-6979:
------------------------------------
Are we ok to get this in? Or it is not important, and let users to deal with it since it just a rare case during upgrade?
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6979) recovered.edits file should not
break distributed log splitting
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478243#comment-13478243 ]
Hudson commented on HBASE-6979:
-------------------------------
Integrated in HBase-TRUNK #3452 (See [https://builds.apache.org/job/HBase-TRUNK/3452/])
HBASE-6979 recovered.edits file should not break distributed log splitting (Revision 1399352)
Result = FAILURE
jxiang :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6979) recovered.edits file should not break
distributed log splitting
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-6979:
--------------------------
Summary: recovered.edits file should not break distributed log splitting (was: recovered.edits file should no break SplitLogManager)
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6979) recovered.edits file should not
break distributed log splitting
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474632#comment-13474632 ]
Jimmy Xiang commented on HBASE-6979:
------------------------------------
Good question. This happens only during upgrade from a very old version. We'd like to fix that so as to make sure users have some good experience with upgrade. The test I added to simulate the error condition to make sure the fix is indeed working.
I thought about delete it. If trash is not enabled, then it is gone. As to /tmp, I am very open. I could not find a better place. I think we can delete
it as well, just not very sure.
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6979) recovered.edits file should not
break distributed log splitting
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477141#comment-13477141 ]
Jimmy Xiang commented on HBASE-6979:
------------------------------------
Thanks a lot for the review. I will commit this in trunk tomorrow if no objection.
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6979) recovered.edits file should not
break distributed log splitting
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478498#comment-13478498 ]
Hudson commented on HBASE-6979:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #225 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/225/])
HBASE-6979 recovered.edits file should not break distributed log splitting (Revision 1399352)
Result = FAILURE
jxiang :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Fix For: 0.96.0
>
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6979) recovered.edits file should not
break distributed log splitting
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13474582#comment-13474582 ]
stack commented on HBASE-6979:
------------------------------
Why we have a file and folder named recovered.edits (We used to have a file named recovered.edits but so we could handle the case where a regionserver has multiple edits to replay, we made a folder named recovered.edits into which we put the files to replay). In tests there should not be any file named recovered.edits. Is the test making these?
That said, the patch is good. Is there a better place than /tmp to move the file? If we delete it, it'll go into hadoop trash? Should stick around a while in there? Would that be good enough?
> recovered.edits file should not break distributed log splitting
> ---------------------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6979) recovered.edits file should no break
SplitLogManager
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-6979:
-------------------------------
Attachment: trunk-6979.patch
> recovered.edits file should no break SplitLogManager
> ----------------------------------------------------
>
> Key: HBASE-6979
> URL: https://issues.apache.org/jira/browse/HBASE-6979
> Project: HBase
> Issue Type: Improvement
> Components: master
> Reporter: Jimmy Xiang
> Assignee: Jimmy Xiang
> Attachments: trunk-6979.patch
>
>
> Distributed log splitting fails in creating the recovered.edits folder during upgrade because there is a file called recovered.edits there.
> Instead of checking if the patch exists, we need to check if it exists and is a path.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira