You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Gary Helmling (JIRA)" <ji...@apache.org> on 2011/09/15 03:00:26 UTC
[jira] [Created] (HBASE-4414) Region splits by size not being
trigger in at least some cases
Region splits by size not being trigger in at least some cases
--------------------------------------------------------------
Key: HBASE-4414
URL: https://issues.apache.org/jira/browse/HBASE-4414
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Gary Helmling
Priority: Blocker
Fix For: 0.92.0
We seem to have lost the triggering of region splits by size somewhere in trunk.
Running a simple test to load data only:
1. create 'usertable', 'f1' in hbase shell
2. run a YCSB load of 10M records
I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
{noformat}
ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
{noformat}
As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling updated HBASE-4414:
---------------------------------
Summary: Region splits by size not being triggered in at least some cases (was: Region splits by size not being trigger in at least some cases)
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105824#comment-13105824 ]
Gary Helmling commented on HBASE-4414:
--------------------------------------
@Stack -- yeah, my reasoning was I didn't want to interrupt the recursive compaction trigger if we were already exceeding the number of blocking store files. We'll see if this is sufficient in practice or if we need something more. In my testing of the load-only case it split out all regions below the max file size as expected.
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch, HBASE-4414_2.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4414) Region splits by size not being
trigger in at least some cases
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling updated HBASE-4414:
---------------------------------
Attachment: HBASE-4414.patch
The attached simple patch causes splits to happen during my load-only test case. But I'm not sure if piggy-backing on the memstore flushes is really the right place to check for this? Anyone more familiar with the compaction code want to comment?
> Region splits by size not being trigger in at least some cases
> --------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling updated HBASE-4414:
---------------------------------
Attachment: HBASE-4414_2.patch
Updated patch that moves the check for a split to following a completed compaction. This seems like the right place to check for it, since the compaction down into a single file is what would have caused us to exceed the region max file size.
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch, HBASE-4414_2.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling reassigned HBASE-4414:
------------------------------------
Assignee: Gary Helmling
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105883#comment-13105883 ]
Hudson commented on HBASE-4414:
-------------------------------
Integrated in HBase-TRUNK #2218 (See [https://builds.apache.org/job/HBase-TRUNK/2218/])
HBASE-4414 Region splits by size not being triggered
garyh :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/CompactionRequest.java
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch, HBASE-4414_2.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105767#comment-13105767 ]
Ted Yu commented on HBASE-4414:
-------------------------------
+1 on patch version 2.
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch, HBASE-4414_2.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105515#comment-13105515 ]
Gary Helmling commented on HBASE-4414:
--------------------------------------
So it looks like this patch isn't quite sufficient yet for splits to always happen by max filesize.
The patch does work during active writes (when we're triggering memstore flushes). In this case, regions get correctly split by size.
But following the initial write load, on an inactive cluster, the multiple store files from flushes get compacted back into a single store file that can then exceed max file size, in the worst case by multiple times. So we still need some kind of split check following compaction to ensure we're not exceeding max size.
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105776#comment-13105776 ]
stack commented on HBASE-4414:
------------------------------
Thats where it used to be before (smile).
You have us go straight back to compacting if it turns out more to compact and priority is high. Thats fine I suppose. If high priority we probably want to go compact again rather than split. We won't know it a problem till we have some experience (we didn't have this recursive compaction notion previously). I'd say go ahead and commit Gary.
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch, HBASE-4414_2.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Helmling resolved HBASE-4414.
----------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Committed to trunk. Thanks for the reviews, Stack and Ted.
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch, HBASE-4414_2.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4414) Region splits by size not being
triggered in at least some cases
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105090#comment-13105090 ]
stack commented on HBASE-4414:
------------------------------
Seems like stuff is not hooked up. We used to check splits after a compaction before. Undoing split following on from a compaction is probably what we want but there is no ongoing split check w/o your patch. Checking on flush seems as good a time as any to look for split.
> Region splits by size not being triggered in at least some cases
> ----------------------------------------------------------------
>
> Key: HBASE-4414
> URL: https://issues.apache.org/jira/browse/HBASE-4414
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.92.0
> Reporter: Gary Helmling
> Priority: Blocker
> Fix For: 0.92.0
>
> Attachments: HBASE-4414.patch
>
>
> We seem to have lost the triggering of region splits by size somewhere in trunk.
> Running a simple test to load data only:
> 1. create 'usertable', 'f1' in hbase shell
> 2. run a YCSB load of 10M records
> I wind up with a single region containing all records, around 13GB, despite max region size being configured to 640MB.
> {noformat}
> ip-10-160-217-155.us-west-1.compute.internal:8120 1316045713501
> requestsPerSecond=0, numberOfOnlineRegions=1, usedHeapMB=1544, maxHeapMB=2962
> usertable,,1316045755455.1e11a9f71072113258942e03dabaa468.
> numberOfStores=1, numberOfStorefiles=16, storefileUncompressedSizeMB=13611, storefileSizeMB=13621, compressionRatio=1.0007, memstoreSizeMB=50, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=1930, rootIndexSizeKB=108, totalStaticIndexSizeKB=10511, totalStaticBloomSizeKB=0, totalCompactingKVs=3356000, currentCompactedKVs=3356000, compactionProgressPct=1.0
> {noformat}
> As best I can tell, the changes introduced in HBASE-3797 and HBASE-1476 dropped some cases where we were triggering region splits when we didn't compact.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira