[jira] [Commented] (HBASE-14978) Don't allow Multi to retain too many blocks

["Anoop Sam John (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-14978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15946482#comment-15946482 ] 

Anoop Sam John commented on HBASE-14978:
----------------------------------------

Reading this part of code again
{code}
if (c instanceof ByteBufferCell) {
          ByteBufferCell bbCell = (ByteBufferCell) c;
          ByteBuffer bb = bbCell.getValueByteBuffer();
          if (bb != lastBlock) {
            context.incrementResponseBlockSize(bb.capacity());
            lastBlock = bb;
          }
        }
{code}
This is specific case when the cell is read from BucketCache. The cell will be backed by DBB in the Bucket Cache.  Each of these BB in bucket cache is of capacity 4 MB!  It is possible to have multiple buckets and blocks within a BB.  The code above will take the response block size as 4 MB!   This will make the response to be send back immediately.  This might not be correct IMO.  If this was counting 64 KB block size ya, that may be ok but 4 MB is too much.  What do you say [~eclark]?
cc [~saint.ack@gmail.com], [~ram_krish]

> Don't allow Multi to retain too many blocks
> -------------------------------------------
>
>                 Key: HBASE-14978
>                 URL: https://issues.apache.org/jira/browse/HBASE-14978
>             Project: HBase
>          Issue Type: Improvement
>          Components: io, IPC/RPC, regionserver
>    Affects Versions: 2.0.0, 1.2.0, 1.3.0
>            Reporter: Elliott Clark
>            Assignee: Elliott Clark
>            Priority: Blocker
>             Fix For: 2.0.0, 1.2.0, 1.3.0
>
>         Attachments: HBASE-14978-branch-1.2.patch, HBASE-14978-branch-1.2.patch, HBASE-14978-branch-1.2.patch, HBASE-14978-branch-1.2.patch, HBASE-14978.patch, HBASE-14978-v1.patch, HBASE-14978-v2.patch, HBASE-14978-v3.patch, HBASE-14978-v4.patch
>
>
> Scans and Multi's have limits on the total size of cells that can be returned. However if those requests are not all pointing at the same blocks then the KeyValues can keep alive a lot more data than their size.
> Take the following example:
> A multi with a list of 10000 gets to a fat row. Each column being returned in in a different block. Each column is small 32 bytes or so.
> So the total cell size will be 32 * 10000 = ~320kb. However if each block is 128k then total retained heap size will be almost 2gigs.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)