You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Ning Li (JIRA)" <ji...@apache.org> on 2008/06/03 20:34:44 UTC
[jira] Created: (HBASE-663) Incorrect sequence number for cache
flush
Incorrect sequence number for cache flush
-----------------------------------------
Key: HBASE-663
URL: https://issues.apache.org/jira/browse/HBASE-663
Project: Hadoop HBase
Issue Type: Bug
Components: regionserver
Reporter: Ning Li
Priority: Minor
An HRegion asks each HStore to flush its cache with a sequence number X. The assumption is that all the updates before X will be flushed. So during the startup reconstruction, the updates before X are skipped.
The use of updatesLock should guarantee that all the updates before X will be flushed when HStore flushes with X - snapshots are taken after the write lock on updatesLock is acquired, while all the updates are written to the log and to the cache with the read lock on updatesLock is acquired. However, because the sequence number X is obtained without the write lock on updatesLock, some updates with sequence number <= X may not have been written to the cache which will be flushed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-663) Incorrect sequence number for cache
flush
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12602770#action_12602770 ]
stack commented on HBASE-663:
-----------------------------
Branch does not have this issue. Setting of sequenceid is done w/i the write lock. No need of backport.
> Incorrect sequence number for cache flush
> -----------------------------------------
>
> Key: HBASE-663
> URL: https://issues.apache.org/jira/browse/HBASE-663
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.2, 0.2.0
> Reporter: Ning Li
> Assignee: Jim Kellerman
> Priority: Blocker
> Fix For: 0.1.3, 0.2.0
>
>
> An HRegion asks each HStore to flush its cache with a sequence number X. The assumption is that all the updates before X will be flushed. So during the startup reconstruction, the updates before X are skipped.
> The use of updatesLock should guarantee that all the updates before X will be flushed when HStore flushes with X - snapshots are taken after the write lock on updatesLock is acquired, while all the updates are written to the log and to the cache with the read lock on updatesLock is acquired. However, because the sequence number X is obtained without the write lock on updatesLock, some updates with sequence number <= X may not have been written to the cache which will be flushed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-663) Incorrect sequence number for cache
flush
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman updated HBASE-663:
--------------------------------
Fix Version/s: 0.2.0
0.1.3
Priority: Blocker (was: Minor)
Affects Version/s: 0.2.0
0.1.2
> Incorrect sequence number for cache flush
> -----------------------------------------
>
> Key: HBASE-663
> URL: https://issues.apache.org/jira/browse/HBASE-663
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.2, 0.2.0
> Reporter: Ning Li
> Assignee: Jim Kellerman
> Priority: Blocker
> Fix For: 0.1.3, 0.2.0
>
>
> An HRegion asks each HStore to flush its cache with a sequence number X. The assumption is that all the updates before X will be flushed. So during the startup reconstruction, the updates before X are skipped.
> The use of updatesLock should guarantee that all the updates before X will be flushed when HStore flushes with X - snapshots are taken after the write lock on updatesLock is acquired, while all the updates are written to the log and to the cache with the read lock on updatesLock is acquired. However, because the sequence number X is obtained without the write lock on updatesLock, some updates with sequence number <= X may not have been written to the cache which will be flushed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-663) Incorrect sequence number for cache
flush
Posted by "Ning Li (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12602019#action_12602019 ]
Ning Li commented on HBASE-663:
-------------------------------
The fix is that in HRegion's internalFlushcache():
Change
this.updatesLock.writeLock().lock();
try {
for (HStore s: stores.values()) {
s.snapshot();
}
} finally {
this.updatesLock.writeLock().unlock();
}
long sequenceId = log.startCacheFlush();
To
this.updatesLock.writeLock().lock();
try {
for (HStore s: stores.values()) {
s.snapshot();
}
long sequenceId = log.startCacheFlush();
} finally {
this.updatesLock.writeLock().unlock();
}
> Incorrect sequence number for cache flush
> -----------------------------------------
>
> Key: HBASE-663
> URL: https://issues.apache.org/jira/browse/HBASE-663
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Reporter: Ning Li
> Priority: Minor
>
> An HRegion asks each HStore to flush its cache with a sequence number X. The assumption is that all the updates before X will be flushed. So during the startup reconstruction, the updates before X are skipped.
> The use of updatesLock should guarantee that all the updates before X will be flushed when HStore flushes with X - snapshots are taken after the write lock on updatesLock is acquired, while all the updates are written to the log and to the cache with the read lock on updatesLock is acquired. However, because the sequence number X is obtained without the write lock on updatesLock, some updates with sequence number <= X may not have been written to the cache which will be flushed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HBASE-663) Incorrect sequence number for cache
flush
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman reassigned HBASE-663:
-----------------------------------
Assignee: Jim Kellerman
> Incorrect sequence number for cache flush
> -----------------------------------------
>
> Key: HBASE-663
> URL: https://issues.apache.org/jira/browse/HBASE-663
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.2, 0.2.0
> Reporter: Ning Li
> Assignee: Jim Kellerman
> Priority: Minor
> Fix For: 0.1.3, 0.2.0
>
>
> An HRegion asks each HStore to flush its cache with a sequence number X. The assumption is that all the updates before X will be flushed. So during the startup reconstruction, the updates before X are skipped.
> The use of updatesLock should guarantee that all the updates before X will be flushed when HStore flushes with X - snapshots are taken after the write lock on updatesLock is acquired, while all the updates are written to the log and to the cache with the read lock on updatesLock is acquired. However, because the sequence number X is obtained without the write lock on updatesLock, some updates with sequence number <= X may not have been written to the cache which will be flushed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-663) Incorrect sequence number for cache
flush
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12602050#action_12602050 ]
stack commented on HBASE-663:
-----------------------------
Thanks for the patch Ning. You hanging around HBase these times? If so, what you working on (Out of interest)?
> Incorrect sequence number for cache flush
> -----------------------------------------
>
> Key: HBASE-663
> URL: https://issues.apache.org/jira/browse/HBASE-663
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.2, 0.2.0
> Reporter: Ning Li
> Assignee: Jim Kellerman
> Priority: Blocker
> Fix For: 0.1.3, 0.2.0
>
>
> An HRegion asks each HStore to flush its cache with a sequence number X. The assumption is that all the updates before X will be flushed. So during the startup reconstruction, the updates before X are skipped.
> The use of updatesLock should guarantee that all the updates before X will be flushed when HStore flushes with X - snapshots are taken after the write lock on updatesLock is acquired, while all the updates are written to the log and to the cache with the read lock on updatesLock is acquired. However, because the sequence number X is obtained without the write lock on updatesLock, some updates with sequence number <= X may not have been written to the cache which will be flushed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-663) Incorrect sequence number for cache
flush
Posted by "Jim Kellerman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman resolved HBASE-663.
---------------------------------
Resolution: Fixed
Committed to branch and trunk.
> Incorrect sequence number for cache flush
> -----------------------------------------
>
> Key: HBASE-663
> URL: https://issues.apache.org/jira/browse/HBASE-663
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.1.2, 0.2.0
> Reporter: Ning Li
> Assignee: Jim Kellerman
> Priority: Blocker
> Fix For: 0.1.3, 0.2.0
>
>
> An HRegion asks each HStore to flush its cache with a sequence number X. The assumption is that all the updates before X will be flushed. So during the startup reconstruction, the updates before X are skipped.
> The use of updatesLock should guarantee that all the updates before X will be flushed when HStore flushes with X - snapshots are taken after the write lock on updatesLock is acquired, while all the updates are written to the log and to the cache with the read lock on updatesLock is acquired. However, because the sequence number X is obtained without the write lock on updatesLock, some updates with sequence number <= X may not have been written to the cache which will be flushed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.