[jira] Commented: (HBASE-2248) Provide new non-copy mechanism to assure atomic reads in get and scan

["ryan rawson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12839964#action_12839964 ] 

ryan rawson commented on HBASE-2248:
------------------------------------

I'm working on this, there is a general approach hammered out and code to be written.

The approach is like so:

- on read from memstore, for each row, we grab the 'read number' and ignore any keyvalues in the structure newer (ie: > value)
- on put to hregion/memstore, we start a write 'tx' and get a write-number, and put keyvalues with said write-number.  when we are finished, that write-number is 'commited' which causes the read number to be advanced most of the time.  under concurrent writes we have a little queue and slower puts may slightly hold up puts that come before it.  

this will need to be extensively tested to see how the performance profile changes. it will allow us to remove the newScannerLock.

> Provide new non-copy mechanism to assure atomic reads in get and scan
> ---------------------------------------------------------------------
>
>                 Key: HBASE-2248
>                 URL: https://issues.apache.org/jira/browse/HBASE-2248
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.3
>            Reporter: Dave Latham
>             Fix For: 0.20.4
>
>         Attachments: HBASE-2248-demonstrate-previous-impl-bugs.patch, hbase-2248.gc, HBASE-2248.patch, Screen shot 2010-02-23 at 10.33.38 AM.png, threads.txt
>
>
> HBASE-2037 introduced a new MemStoreScanner which triggers a ConcurrentSkipListMap.buildFromSorted clone of the memstore and snapshot when starting a scan.
> After upgrading to 0.20.3, we noticed a big slowdown in our use of short scans.  Some of our data repesent a time series.   The data is stored in time series order, MR jobs often insert/update new data at the end of the series, and queries usually have to pick up some or all of the series.  These are often scans of 0-100 rows at a time.  To load one page, we'll observe about 20 such scans being triggered concurrently, and they take 2 seconds to complete.  Doing a thread dump of a region server shows many threads in ConcurrentSkipListMap.biuldFromSorted which traverses the entire map of key values to copy it.  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.