You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "chunhui shen (JIRA)" <ji...@apache.org> on 2012/07/06 08:07:34 UTC
[jira] [Created] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
chunhui shen created HBASE-6337:
-----------------------------------
Summary: [MTTR] Remove renaming tmp log file in SplitLogManager
Key: HBASE-6337
URL: https://issues.apache.org/jira/browse/HBASE-6337
Project: HBase
Issue Type: Bug
Reporter: chunhui shen
Assignee: chunhui shen
As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410965#comment-13410965 ]
Zhihong Ted Yu commented on HBASE-6337:
---------------------------------------
I am running test suite for 0.94
Will report back if there is any test failure.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409149#comment-13409149 ]
chunhui shen commented on HBASE-6337:
-------------------------------------
@ted
With the patch, we will split log to region's directory directory like master-local-log-splitting, so we needn't to call moveRecoveredEditsFromTemp any more, the code
{code}
Path dst = ZKSplitLog.stripSplitLogTempDir(rootdir, src);
if (ZKSplitLog.isCorruptFlagFile(dst)) {
continue;
}
{code} is used to rename tmp log file, and we will judge whether the log file is corrupted in new method
{code}
if (ZKSplitLog.isCorrupted(rootdir, logPath.getName(), fs)) {
+ corruptedLogs.add(logPath);
+ } else {
+ processedLogs.add(logPath);
+ }
{code}
@stack
HLogSplitter.splitLogFileToTemp will be called in many tests, in order to quick solution, I just pass null rather than remove the tmp arg.
With the patch v2, I run the tests and passed in 0.94 version.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Attachments: HBASE-6337v1.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6337:
---------------------------------
Fix Version/s: (was: 0.94.2)
0.94.1
Ship it.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
chunhui shen updated HBASE-6337:
--------------------------------
Attachment: HBASE-6337v2.patch
Updating patch v2:
As stack's comment, removing tmp arg in method HLogSplitter#splitLogFile;
And removing some methods won't be called after the patch.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409929#comment-13409929 ]
Lars Hofhansl commented on HBASE-6337:
--------------------------------------
Change itself looks to me. If the writing directly into recoverededits is not a problem: +1
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428723#comment-13428723 ]
Hudson commented on HBASE-6337:
-------------------------------
Integrated in HBase-0.94-security-on-Hadoop-23 #6 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/6/])
HBASE-6337 [MTTR] Remove renaming tmp log file in SplitLogManager (Chunhui) (Revision 1359959)
Result = FAILURE
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6337) [MTTR] Remove renaming tmp log file in
SplitLogManager
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl closed HBASE-6337.
--------------------------------
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.94.1, 0.96.0
>
> Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409199#comment-13409199 ]
Hadoop QA commented on HBASE-6337:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12535622/HBASE-6337v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2349//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2349//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2349//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2349//console
This message is automatically generated.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410239#comment-13410239 ]
Hadoop QA commented on HBASE-6337:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12535776/HBASE-6337v4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2355//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2355//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2355//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2355//console
This message is automatically generated.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408046#comment-13408046 ]
Zhihong Ted Yu commented on HBASE-6337:
---------------------------------------
Where does the following code from moveRecoveredEditsFromTemp() go ?
{code}
Path dst = ZKSplitLog.stripSplitLogTempDir(rootdir, src);
if (ZKSplitLog.isCorruptFlagFile(dst)) {
continue;
}
{code}
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Attachments: HBASE-6337v1.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
chunhui shen updated HBASE-6337:
--------------------------------
Attachment: HBASE-6337v4.patch
Uploading patch v4 : Rename method name moveSplitLogFile to finishSplitLogFile
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409564#comment-13409564 ]
stack commented on HBASE-6337:
------------------------------
@Chunhui The patch looks good. Thanks for all the cleanup. moveSplitLogFile could be named better since now there is no actually moving being done. Call it finishSplitLog or something? Have you checked out the open region side of the affair to see if any conditions under which we might bungle the replay of recovered.edits files? I don't think it possible as long as split finishes successfully before region opens (or we crash out the cluster if we can't split).
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6337:
----------------------------------
Status: Open (was: Patch Available)
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409164#comment-13409164 ]
Zhihong Ted Yu commented on HBASE-6337:
---------------------------------------
For patch v2, I got a test failure:
{code}
testSplitLogFileFirstLineCorruptionLog(org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit) Time elapsed: 1.456 sec <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<0>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit.testSplitLogFileFirstLineCorruptionLog(TestHLogSplit.java:1108)
{code}
See if you can reproduce it.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410864#comment-13410864 ]
Zhihong Ted Yu commented on HBASE-6337:
---------------------------------------
I ran TestHLog with patch v4 3 times and they passed on MacBook.
It is a pity that builds on PreCommit-HBASE-Build were truncated to July 4th - the first link above is no longer working.
Looks like Chunhui has answered all the questions.
The 0.94 equivalent of the patch was running in their dev cluster.
Will wait for 24 hours for further comments.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407730#comment-13407730 ]
chunhui shen commented on HBASE-6337:
-------------------------------------
I will upload a patch later as a start.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410011#comment-13410011 ]
chunhui shen commented on HBASE-6337:
-------------------------------------
bq.Have you checked out the open region side of the affair to see if any conditions under which we might bungle the replay of recovered.edits files?
I think it won't happen, when splitting log, the edits file is named ends with ".temp", and repalying edits will skip these files.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409190#comment-13409190 ]
Lars Hofhansl commented on HBASE-6337:
--------------------------------------
Why was the rename done in the first place? Was it so that a RegionServer does not see partially written files in .recoverededits? If that is the case, is this still safe? (Not at a computer right now, so it's possible I misunderstood the nature of the change)
Also, didn't realize you had a 0.94 patch already. Added 0.94.2 as target.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6337:
---------------------------------
Fix Version/s: 0.94.2
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409158#comment-13409158 ]
Zhihong Ted Yu commented on HBASE-6337:
---------------------------------------
Thanks for the quick response.
{code}
+ HLogSplitter.moveSplittedLogFile(hlogDir, oldLogDir, logfile.getPath()
{code}
The past tense for verb split is split. Please remove 'ted' before Logfile in the above method name.
If you can put the 0.94 patch onto a real cluster and collect performance numbers, that would be great.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411087#comment-13411087 ]
Zhihong Ted Yu commented on HBASE-6337:
---------------------------------------
Ran 0.94 test suite again.
The tests passed.
Integrated to 0.94 and trunk.
Thanks for the patch, Chunhui.
Thanks for the review, Stack, Ram and Lars.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409557#comment-13409557 ]
stack commented on HBASE-6337:
------------------------------
@Lars I'm not sure why it was done originally. Prakash was probably being cautious. As long as a region does not open before log split completes, as Chunhui says, I think we should be fine. The code as is already overwrites recovered.edits files with same sequence number (which could happen if first split failed and we then rerun it before region open). Maybe there are failure scenarios we've not yet encountered or imagined -- have you tried conjure any new ones Chunhui? And don't we have tests for some common failures already? -- but this approach should be fine I believe.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410945#comment-13410945 ]
Lars Hofhansl commented on HBASE-6337:
--------------------------------------
+1 on patch, +1 for 0.94 (can pull back into 0.94.1)
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13408344#comment-13408344 ]
stack commented on HBASE-6337:
------------------------------
If you are removing tmp dir, don't be half-hearted about it. Go for it Chunhui. So, for example here:
{code}
+ if (HLogSplitter.splitLogFileToTemp(rootdir, null,
{code}
... don't pass mull but remove the tmp arg. altogether (unless you have a reason to retain it -- you seem to remove tmp dir elsewhere in your patch).
Is that all it takes?
Does this pass tests? What about the tests that replay recovered edits. These are passing?
Good stuff Chunhui.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Attachments: HBASE-6337v1.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
chunhui shen updated HBASE-6337:
--------------------------------
Attachment: HBASE-6337v3.patch
Updating patch v3 with Ted's comment and fix the mistake in the test when making patch for trunk from 0.94.
I run all the tests and passed in 0.94
@Ted
Could you run it by HadoopQA
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410997#comment-13410997 ]
Zhihong Ted Yu commented on HBASE-6337:
---------------------------------------
Here is test result for 0.94:
{code}
Tests run: 648, Failures: 0, Errors: 0, Skipped: 2
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30:14.639s
[INFO] Finished at: Tue Jul 10 15:16:19 GMT-07:00 2012
{code}
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl updated HBASE-6337:
---------------------------------
Fix Version/s: 0.96.0
Let's try for 0.96 for now.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409182#comment-13409182 ]
chunhui shen commented on HBASE-6337:
-------------------------------------
bq.If you can put the 0.94 patch onto a real cluster and collect performance numbers, that would be great.
If the distributed-log-splitting's bottleneck is not on the SplitLogManager, this improvement is just reducing NN operations, else it will decrease the time of distributed-log-splitting.
We have test for the case:
9 regionserver, 2500 regions per hlog file, total 280 hlog files,16 ms per rename operation
With the patch, reduce the time of distributed-log-splitting from 3+ hours to 20+ mins
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6337:
----------------------------------
Attachment: 6337-94.txt
Patch for 0.94
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6337:
----------------------------------
Status: Patch Available (was: Open)
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411165#comment-13411165 ]
Hudson commented on HBASE-6337:
-------------------------------
Integrated in HBase-0.94-security #39 (See [https://builds.apache.org/job/HBase-0.94-security/39/])
HBASE-6337 [MTTR] Remove renaming tmp log file in SplitLogManager (Chunhui) (Revision 1359959)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409196#comment-13409196 ]
chunhui shen commented on HBASE-6337:
-------------------------------------
bq.Why was the rename done in the first place?
We don't split log file to tmp directory anymore, and SplitLogManager need't to rename the tmp recovered.edits files when finishing task.
bq.If that is the case, is this still safe?
How unsafe happen? Region will replay edits only in initialization, so we must alreay split the logs, otherwise we won't assign the region.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409221#comment-13409221 ]
chunhui shen commented on HBASE-6337:
-------------------------------------
@ram
bq.If the region dir is deleted by CJ then in the current patch we will get IOException and splitlog may be retried?
For this case, if region dir is deleted by CJ when we are appending data, we will get IOException and splitlog would retry. In the next retry splitting, we won't split log for this region through the following code:
HLogSplitter#getRegionSplitEditsPath
{code}
if (!fs.exists(regiondir)) {
LOG.info("This region's directory doesn't exist: "
+ regiondir.toString() + ". It is very likely that it was" +
" already split so it's safe to discard those edits.");
return null;
}
{code}
bq.They were doing listAll and expecting a set of files? Now we are not doing that? Is it ok?
This operation is used to list all the recovered.edits files in the tmp directory and move them to region'dir, however now we don't split log to tmp directory, so no need to do that.
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "chunhui shen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
chunhui shen updated HBASE-6337:
--------------------------------
Attachment: HBASE-6337v1.patch
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Attachments: HBASE-6337v1.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409214#comment-13409214 ]
ramkrishna.s.vasudevan commented on HBASE-6337:
-----------------------------------------------
@Chunhui
If the region dir is deleted by CJ then in the current patch we will get IOException and splitlog may be retried? HBASE-6050 was one such situation.
In the current code
{code}
List<FileStatus> files = listAll(fs, stagingDir);
{code}
They were doing listAll and expecting a set of files? Now we are not doing that? Is it ok?
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.2
>
> Attachments: HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6337) [MTTR] Remove renaming tmp log file
in SplitLogManager
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl resolved HBASE-6337.
----------------------------------
Resolution: Fixed
> [MTTR] Remove renaming tmp log file in SplitLogManager
> -------------------------------------------------------
>
> Key: HBASE-6337
> URL: https://issues.apache.org/jira/browse/HBASE-6337
> Project: HBase
> Issue Type: Bug
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6337-94.txt, HBASE-6337v1.patch, HBASE-6337v2.patch, HBASE-6337v3.patch, HBASE-6337v4.patch
>
>
> As HBASE-6309 mentioned, we also encounter problem of distributed-log-splitting take much more time than matser-local-log-splitting because lots of SplitLogManager 's renaming operations when finishing task.
> Could we try to remove renaming tmp log file in SplitLogManager through splitting log to regions' recover.edits directory directly as the same as the master-local-log-splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira