[jira] [Created] (PHOENIX-2189) Starting from HBase 1.x, phoenix shouldn't probably override the hbase.client.scanner.caching attribute

["Samarth Jain (JIRA)" <ji...@apache.org>]

Samarth Jain created PHOENIX-2189:
-------------------------------------

             Summary: Starting from HBase 1.x, phoenix shouldn't probably override the hbase.client.scanner.caching attribute
                 Key: PHOENIX-2189
                 URL: https://issues.apache.org/jira/browse/PHOENIX-2189
             Project: Phoenix
          Issue Type: Bug
            Reporter: Samarth Jain


After PHOENIX-2188 is fixed, we need to think about whether it makes sense to override the scanner cache size in Phoenix for branches HBase 1.x. For ex  - in HBase 1.1, the default value of hbase.client.scanner.caching is now Integer.MAX_VALUE.

{code:xml}
<property>
    <name>hbase.client.scanner.caching</name>
    <value>2147483647</value>
    <description>Number of rows that we try to fetch when calling next
    on a scanner if it is not served from (local, client) memory. This configuration
    works together with hbase.client.scanner.max.result.size to try and use the
    network efficiently. The default value is Integer.MAX_VALUE by default so that
    the network will fill the chunk size defined by hbase.client.scanner.max.result.size
    rather than be limited by a particular number of rows since the size of rows varies
    table to table. If you know ahead of time that you will not require more than a certain
    number of rows from a scan, this configuration should be set to that row limit via
    Scan#setCaching. Higher caching values will enable faster scanners but will eat up more
    memory and some calls of next may take longer and longer times when the cache is empty.
    Do not set this value such that the time between invocations is greater than the scanner
    timeout; i.e. hbase.client.scanner.timeout.period</description>
  </property>
{code:xml}

>From the comments it sounds like, by default, HBase is going to provide an upper bound on the scanner cache size in bytes and not number of records. 

If we end up overriding the hbase.client.scanner.caching to 1000, then potentially for narrower rows we will likely be fetching too few rows. For wider rows, likely the bytes limit will kick in to make sure we don't end up caching too much on the client.

Maybe we shouldn't be using the scanner caching override at all? Thoughts? [~jamestaylor], [~lhofhansl]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)