[jira] Commented: (HBASE-469) Streamline HStore startup and compactions

["Jim Kellerman (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584201#action_12584201 ] 

Jim Kellerman commented on HBASE-469:
-------------------------------------

> stack - 31/Mar/08 02:26 PM
> Before reviewing, does this new patch do as previous versions did - IIRC - removing the memory cap
> for a region rather moving the memory cap down to the store such that if ten stores, we could use ten
> times the memory we used to use if all stores are undergoing updates?

No the flushing policy has not changed. That is it still uses checkResources to block updates when the total memcache size for a region reaches 64MB.

The patch for HBASE-69 is the one you're thinking of.

This patch makes the HStore smarter. For example loadHStoreFiles also gets the sequence number and file size while it is loading the stores so a separate scan is no longer necessary. cache flushing updates the store size. The only time the store size is recomputed is after a compaction: in that case compact knows the size of the file it just created so it only has to look at any new store files that might have been created by a cache flush that occurred while the compaction was running.

compact also computes the mid key if the store needs to be (and can be) split even if it decides no compaction is necessary. Thus the compactor thread knows if the region needs to be split after calling compact and does not have to query the region.

QueueEntry's were removed because neither the flusher nor the compactor use delay queues and at that point, the only payload of the QueueEntry was the region reference. HRegion now implements equals and hashCode so it can be inserted into a blocking queue and a hash set. The hash set is a set of all the regions in the queue and can determine if a region is in the queue much faster than BlockingQueue.contains() which does a linear scan of the queue.


> Streamline HStore startup and compactions
> -----------------------------------------
>
>                 Key: HBASE-469
>                 URL: https://issues.apache.org/jira/browse/HBASE-469
>             Project: Hadoop HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.2.0
>            Reporter: Jim Kellerman
>            Assignee: Jim Kellerman
>             Fix For: 0.2.0
>
>         Attachments: patch.txt, patch.txt
>
>
> Several useful patches that streamline HStore startup and compactions that were a part of the abandoned changes in HBASE-69 should be incorporated.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.