You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2011/03/28 23:48:05 UTC
[jira] [Created] (HBASE-3707) Flush memstore after a configurable
number of inserts not simply based on size
Flush memstore after a configurable number of inserts not simply based on size
------------------------------------------------------------------------------
Key: HBASE-3707
URL: https://issues.apache.org/jira/browse/HBASE-3707
Project: HBase
Issue Type: Improvement
Reporter: Andrew Purtell
Priority: Minor
Attachments: HBASE-3707.patch
Memstore upsert performance may be impacted by having a large number of values in the map. Consider flushing the store after a configurable number of inserts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3707) Flush memstore after a configurable
number of inserts not simply based on size
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-3707:
----------------------------------
Attachment: HBASE-3707.patch
Just sketching out an idea here. Using this and ROWCOL blooms helps an upsert-bound use case but there is still something else going on with that.
> Flush memstore after a configurable number of inserts not simply based on size
> ------------------------------------------------------------------------------
>
> Key: HBASE-3707
> URL: https://issues.apache.org/jira/browse/HBASE-3707
> Project: HBase
> Issue Type: Improvement
> Reporter: Andrew Purtell
> Priority: Minor
> Attachments: HBASE-3707.patch
>
>
> Memstore upsert performance may be impacted by having a large number of values in the map. Consider flushing the store after a configurable number of inserts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3707) Flush memstore after a configurable
number of inserts not simply based on size
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-3707:
----------------------------------
Priority: Major (was: Minor)
Maybe this shouldn't be minor priority.
> Flush memstore after a configurable number of inserts not simply based on size
> ------------------------------------------------------------------------------
>
> Key: HBASE-3707
> URL: https://issues.apache.org/jira/browse/HBASE-3707
> Project: HBase
> Issue Type: Improvement
> Reporter: Andrew Purtell
> Attachments: HBASE-3707.patch
>
>
> Memstore upsert performance may be impacted by having a large number of values in the map. Consider flushing the store after a configurable number of inserts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3707) Flush memstore after a configurable
number of inserts not simply based on size
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13012286#comment-13012286 ]
Andrew Purtell commented on HBASE-3707:
---------------------------------------
Running this under the profiler I find this change keeps the time for Store#upsert proportional to that for HRegion#getLastIncrement, at about 30% each for an Increment only workload. Otherwise the relative time for getLastIncrement creeps up to about 70%.
> Flush memstore after a configurable number of inserts not simply based on size
> ------------------------------------------------------------------------------
>
> Key: HBASE-3707
> URL: https://issues.apache.org/jira/browse/HBASE-3707
> Project: HBase
> Issue Type: Improvement
> Reporter: Andrew Purtell
> Priority: Minor
> Attachments: HBASE-3707.patch
>
>
> Memstore upsert performance may be impacted by having a large number of values in the map. Consider flushing the store after a configurable number of inserts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3707) Flush memstore after a configurable
number of inserts not simply based on size
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014741#comment-13014741 ]
Andrew Purtell commented on HBASE-3707:
---------------------------------------
Yes unbounded by default.
Let me look at per-CF.
Agree it is a band aid. I don't suggest applying the patch, just keeping it around if use of it turns out to be expedient somehow.
> Flush memstore after a configurable number of inserts not simply based on size
> ------------------------------------------------------------------------------
>
> Key: HBASE-3707
> URL: https://issues.apache.org/jira/browse/HBASE-3707
> Project: HBase
> Issue Type: Improvement
> Reporter: Andrew Purtell
> Attachments: HBASE-3707.patch
>
>
> Memstore upsert performance may be impacted by having a large number of values in the map. Consider flushing the store after a configurable number of inserts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3707) Flush memstore after a configurable
number of inserts not simply based on size
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014738#comment-13014738 ]
stack commented on HBASE-3707:
------------------------------
Patch looks fine Andrew but does seem like a bandaid for the issue you are seeing with CSLM. You thinking of applying it with unbounded number for the default. You'd add config. on the CF-level for setting max?
> Flush memstore after a configurable number of inserts not simply based on size
> ------------------------------------------------------------------------------
>
> Key: HBASE-3707
> URL: https://issues.apache.org/jira/browse/HBASE-3707
> Project: HBase
> Issue Type: Improvement
> Reporter: Andrew Purtell
> Attachments: HBASE-3707.patch
>
>
> Memstore upsert performance may be impacted by having a large number of values in the map. Consider flushing the store after a configurable number of inserts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira