You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ted Yu (JIRA)" <ji...@apache.org> on 2013/01/03 18:30:13 UTC
[jira] [Updated] (HBASE-7487) Test for split edit file ordering
using failover scenario
[ https://issues.apache.org/jira/browse/HBASE-7487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-7487:
--------------------------
Description:
TestHLogMethods#testGetSplitEditFilesSorted tests the ordering of split edit files. However, we should have a test covering more realistic scenario.
Suppose there are two split edit files for one region.
Their edits are as the following
File 1:
edit id 1001
edit id 1002
edit id 1003
File 2:
edit id 2001
edit id 2002
edit id 2003
As the current logic, file 1 is named "1003" and file 2 is named "2003".
Suppose region's seq id is 1000 before replaying edits files.
So we will replay all the edits in File 1 and File 2
However, if we replay File 2 first, the following case could result in data loss:
1.Completing File 2 replay.
2.Do the internalFlushcache. See HRegion#replayRecoveredEdits:
if (flush) internalFlushcache(null, currentEditSeqId, status);
3.the regionserver crashes
4.Region is assigned to another server, and its seq ID becomes 2003
5.File1 will be skipped when replaying edits
was:
TestHLogMethods#testGetSplitEditFilesSorted tests the ordering of split edit files. However, we should have a test covering more realistic scenario.
Suppose there are two split edit files for one region.
Their edits are as the following
File 1:
edit id 1001
edit id 1002
edit id 1003
File 2:
edit id 2001
edit id 2002
edit id 2003
As the current logic, file 1 is named "1003" and file 2 is named "2003".
Suppose region's seq id is 1000 before replaying edits files.
So we will replay all the edits in File 1 and File 2
However, if we replay File 2 first, the following case could result in data loss:
1.Completing File 2 replay.
2.Do the internalFlushcache.(see HRegion#replayRecoveredEdits:
if (flush) internalFlushcache(null, currentEditSeqId, status);)
3.the regionserver crashes
4.Region is assigned to another server, an its seq ID becomes 2003
5.File1 will be skipped when replaying edits
> Test for split edit file ordering using failover scenario
> ---------------------------------------------------------
>
> Key: HBASE-7487
> URL: https://issues.apache.org/jira/browse/HBASE-7487
> Project: HBase
> Issue Type: Sub-task
> Reporter: Ted Yu
> Fix For: 0.96.0
>
>
> TestHLogMethods#testGetSplitEditFilesSorted tests the ordering of split edit files. However, we should have a test covering more realistic scenario.
> Suppose there are two split edit files for one region.
>
> Their edits are as the following
>
> File 1:
> edit id 1001
> edit id 1002
> edit id 1003
>
> File 2:
> edit id 2001
> edit id 2002
> edit id 2003
>
> As the current logic, file 1 is named "1003" and file 2 is named "2003".
> Suppose region's seq id is 1000 before replaying edits files.
>
> So we will replay all the edits in File 1 and File 2
>
> However, if we replay File 2 first, the following case could result in data loss:
> 1.Completing File 2 replay.
> 2.Do the internalFlushcache. See HRegion#replayRecoveredEdits:
> if (flush) internalFlushcache(null, currentEditSeqId, status);
> 3.the regionserver crashes
> 4.Region is assigned to another server, and its seq ID becomes 2003
> 5.File1 will be skipped when replaying edits
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira