Re: Review Request: HBASE-3160: Use more intelligent priorities for PriorityCompactionQueue

[st...@duboce.net]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1103/#review1695
-----------------------------------------------------------


Looks good to me.  You fellas think it a bug fix or a feature?  Should I punt to 0.92 or include now in TRUNK?


trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java
<http://review.cloudera.org/r/1103/#comment5605>

    Why would we do this -- fiddle w/ our priority if we are going to split?  Should we check if we are to split first and if not then mess w/ priority?



trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
<http://review.cloudera.org/r/1103/#comment5606>

    So, we aggregate these Store counts to come up w/ a Region count and the Region w/ most storefiles gets highest compaction priority?



trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestPriorityCompactionQueue.java
<http://review.cloudera.org/r/1103/#comment5607>

    There is white space here on end of lines... I can fix on commit.


- stack


On 2010-10-28 14:48:04, Nicolas wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.cloudera.org/r/1103/
> -----------------------------------------------------------
> 
> (Updated 2010-10-28 14:48:04)
> 
> 
> Review request for hbase.
> 
> 
> Summary
> -------
> 
> Switch to more intelligent priority metric: blockingSize - max(len(s.storefiles) for s in region.stores) . This will allow us to better prioritize, give us faster responsiveness to users, and feel more cavalier about issuing new compaction requests.  Note that we also found/fixed a major compaction downgrade bug while writing this code.
> 
> 
> This addresses bug HBASE-3160.
>     http://issues.apache.org/jira/browse/HBASE-3160
> 
> 
> Diffs
> -----
> 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1027787 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactionRequestor.java 1027787 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1027787 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1027787 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java 1027787 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/PriorityCompactionQueue.java 1027787 
>   trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1027787 
>   trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestPriorityCompactionQueue.java 1027787 
> 
> Diff: http://review.cloudera.org/r/1103/diff
> 
> 
> Testing
> -------
> 
> mvn clean install -Dtest=TestPriorityCompaction
> dev cluster tests (on 0.89)
> 
> 
> Thanks,
> 
> Nicolas
> 
>