[jira] Commented: (HBASE-910) Scanner misses columns / rows when the scanner is obtained durring a memcache flush

["Jim Kellerman (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12640877#action_12640877 ] 

Jim Kellerman commented on HBASE-910:
-------------------------------------

When a cache flush starts, it uses the current state of the cache as a snapshot, and creates a new cache so that updates will not be blocked during the flush.

A scanner can (and does) use that snapshot until the snapshot is fully written to disk. When the flush is complete the snapshot is deleted to reclaim memory. If the scanner had not yet reached the end of the snapshot before it is deleted, Some rows may be missed in the interval between the flush completes and the scanner is made aware of the new file that was just created.

Closing this window would be very difficult for a couple of reasons:
- holding on to the snapshot until the scanner is done would increase memory pressure on the region server which could lead to the region server running out of memory
- it is inherently difficult to close the window between cache flush completion and notification of the scanner as each is running in a different thread. Increasing synchronization would lead to an overall degredation in performance.

It is my impression that Bigtable does not allow client reads to "see" concurrent updates as each row in the memtable is copy on write so that reads and writes proceed in parallel.

In short, this is a very thorny issue, and HBase is much better in this respect than it was. Scanners used to get a view of the data as it was when the scanner was taken out which did not include data from flushes in progress and did not pick up new files created by cache flushes (at all) when the flush completed. Fixing this issue would require a significant amount of work.

> Scanner misses columns / rows when the scanner is obtained durring a memcache flush
> -----------------------------------------------------------------------------------
>
>                 Key: HBASE-910
>                 URL: https://issues.apache.org/jira/browse/HBASE-910
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>         Environment: latest trunk
>            Reporter: Clint Morgan
>            Priority: Critical
>         Attachments: hbase-910.patch
>
>
> I first noticed that some columns for a row were missing if they are coming from a scanner that was obtained while a memecache flush on the region was in progress. I tried to write a simple unit test to reproduce, however the problem I get in the unit test is that some rows are being missed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.