You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (Created) (JIRA)" <ji...@apache.org> on 2012/02/13 19:59:00 UTC
[jira] [Created] (HBASE-5393) Consider splitting after flushing
Consider splitting after flushing
---------------------------------
Key: HBASE-5393
URL: https://issues.apache.org/jira/browse/HBASE-5393
Project: HBase
Issue Type: Improvement
Affects Versions: 0.90.5
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.94.0
Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.
Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.
I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5393) Consider splitting after flushing
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211130#comment-13211130 ]
Hudson commented on HBASE-5393:
-------------------------------
Integrated in HBase-TRUNK #2665 (See [https://builds.apache.org/job/HBase-TRUNK/2665/])
HBASE-5393 Consider splitting after flushing (Revision 1245727)
Result = FAILURE
jdcryans :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
> Consider splitting after flushing
> ---------------------------------
>
> Key: HBASE-5393
> URL: https://issues.apache.org/jira/browse/HBASE-5393
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.90.5
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-2375-flush-split.patch
>
>
> Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.
> Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.
> I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5393) Consider splitting after flushing
Posted by "Jean-Daniel Cryans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-5393:
--------------------------------------
Attachment: HBASE-2375-flush-split.patch
Patch that I posted in HBASE-2375 that Stack +1'd.
> Consider splitting after flushing
> ---------------------------------
>
> Key: HBASE-5393
> URL: https://issues.apache.org/jira/browse/HBASE-5393
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.90.5
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.94.0
>
> Attachments: HBASE-2375-flush-split.patch
>
>
> Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.
> Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.
> I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5393) Consider splitting after flushing
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212143#comment-13212143 ]
Hudson commented on HBASE-5393:
-------------------------------
Integrated in HBase-0.92-security #91 (See [https://builds.apache.org/job/HBase-0.92-security/91/])
HBASE-5393 Consider splitting after flushing (Revision 1245726)
Result = FAILURE
jdcryans :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
> Consider splitting after flushing
> ---------------------------------
>
> Key: HBASE-5393
> URL: https://issues.apache.org/jira/browse/HBASE-5393
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.90.5
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-2375-flush-split.patch
>
>
> Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.
> Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.
> I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5393) Consider splitting after flushing
Posted by "Jean-Daniel Cryans (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-5393.
---------------------------------------
Resolution: Fixed
Fix Version/s: 0.92.1
Hadoop Flags: Reviewed
Committed to trunk and 0.92, thanks for the votes and reviews guys.
> Consider splitting after flushing
> ---------------------------------
>
> Key: HBASE-5393
> URL: https://issues.apache.org/jira/browse/HBASE-5393
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.90.5
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-2375-flush-split.patch
>
>
> Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.
> Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.
> I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5393) Consider splitting after flushing
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210819#comment-13210819 ]
Hudson commented on HBASE-5393:
-------------------------------
Integrated in HBase-TRUNK-security #114 (See [https://builds.apache.org/job/HBase-TRUNK-security/114/])
HBASE-5393 Consider splitting after flushing (Revision 1245727)
Result = FAILURE
jdcryans :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
> Consider splitting after flushing
> ---------------------------------
>
> Key: HBASE-5393
> URL: https://issues.apache.org/jira/browse/HBASE-5393
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.90.5
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-2375-flush-split.patch
>
>
> Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.
> Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.
> I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5393) Consider splitting after flushing
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208762#comment-13208762 ]
Lars Hofhansl commented on HBASE-5393:
--------------------------------------
+1 on patch and +1 on 0.92.x
> Consider splitting after flushing
> ---------------------------------
>
> Key: HBASE-5393
> URL: https://issues.apache.org/jira/browse/HBASE-5393
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.90.5
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.94.0
>
> Attachments: HBASE-2375-flush-split.patch
>
>
> Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.
> Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.
> I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5393) Consider splitting after flushing
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207081#comment-13207081 ]
stack commented on HBASE-5393:
------------------------------
+1 on putting in 0.92 too...
> Consider splitting after flushing
> ---------------------------------
>
> Key: HBASE-5393
> URL: https://issues.apache.org/jira/browse/HBASE-5393
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.90.5
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.94.0
>
> Attachments: HBASE-2375-flush-split.patch
>
>
> Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.
> Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.
> I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5393) Consider splitting after flushing
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210637#comment-13210637 ]
Hudson commented on HBASE-5393:
-------------------------------
Integrated in HBase-0.92 #287 (See [https://builds.apache.org/job/HBase-0.92/287/])
HBASE-5393 Consider splitting after flushing (Revision 1245726)
Result = SUCCESS
jdcryans :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java
> Consider splitting after flushing
> ---------------------------------
>
> Key: HBASE-5393
> URL: https://issues.apache.org/jira/browse/HBASE-5393
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 0.90.5
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Fix For: 0.94.0, 0.92.1
>
> Attachments: HBASE-2375-flush-split.patch
>
>
> Spawning this from HBASE-2375, I saw that it was much more efficient compaction-wise to check if we can split right after flushing. Much like the ideas that Jon spelled out in the description of that jira, the window is smaller because you don't have to compact and then split right away to only compact again when the daughters open.
> Another thing it improves is while we're normally waiting for the compaction to happen, data that's still coming in will make us go way past the MAX_FILESIZE to a point where for the first region I was seeing a store size 3-4x bigger before it was able to split.
> I targeted this for 0.94, but I'd like to get this into 0.92.1 or .2 too.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira