You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Lars Hofhansl (JIRA)" <ji...@apache.org> on 2012/12/05 07:08:59 UTC
[jira] [Comment Edited] (HBASE-7279) Avoid copying the rowkey in
RegionScanner, StoreScanner, and ScanQueryMatcher
[ https://issues.apache.org/jira/browse/HBASE-7279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510295#comment-13510295 ]
Lars Hofhansl edited comment on HBASE-7279 at 12/5/12 6:07 AM:
---------------------------------------------------------------
[~saint.ack@gmail.com] You mean leave out the timestamp cache, or leave out the change that removes the timestamp cache? :) I can go either way.
However, 8 bytes is not insignificant (the rest of a KV is just 16 + 24 + 4 + 4 + 4 + 8 = 52). (makes me want to remove the keyLength cache as well for another 4 bytes)
At Salesforce we're doing some scans over close to 1bn rows/kvs (most of which won't be shipped to the client).
The issue with the timestamp cache is that it will use 8 bytes, whether we cache anything or not. Over the 1bn KVs we'll produce 8GB of garbage just for this cache.
I would like to put this into 0.94 as well.
was (Author: lhofhansl):
You leave out the timestamp cache, or leave out the change that removes the timestamp cache? :) I can go either way.
However, 8 bytes is not insignificant (the rest of a KV is just 16 + 24 + 4 + 4 + 4 + 8 = 52). (makes me want to remove the keyLength cache as well for another 4 bytes)
At Salesforce we're doing some scans over close to 1bn rows/kvs (most of which won't be shipped to the client).
The issue with the timestamp cache is that it will use 8 bytes, whether we cache anything or not. Over the 1bn KVs we'll produce 8GB of garbage just for this cache.
I would like to put this into 0.94 as well.
> Avoid copying the rowkey in RegionScanner, StoreScanner, and ScanQueryMatcher
> -----------------------------------------------------------------------------
>
> Key: HBASE-7279
> URL: https://issues.apache.org/jira/browse/HBASE-7279
> Project: HBase
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Fix For: 0.96.0, 0.94.4
>
> Attachments: 7279-0.94.txt
>
>
> Did some profiling again.
> I we can gain some performance [1] when passing buffer, rowoffset, and rowlength instead of making a copy of the row key.
> That way we can also remove the row key caching (and this patch also removes the timestamps caching). Considering the sheer number in which we create KVs, every byte save is good.
> [1] (15-20% when data is in the block cache we setup a Filter such that only a single row is returned to the client).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira