[jira] [Created] (PHOENIX-5987) PointLookup may cost too much memory

["Chen Feng (Jira)" <ji...@apache.org>]

Chen Feng created PHOENIX-5987:
----------------------------------

             Summary: PointLookup may cost too much memory
                 Key: PHOENIX-5987
                 URL: https://issues.apache.org/jira/browse/PHOENIX-5987
             Project: Phoenix
          Issue Type: Bug
            Reporter: Chen Feng


When all rowkeys are covered in where conditions, Phoenix use point look up to switch "a huge range scan" to "multi single key scans".
However, the number single key scans are too huge, it quick exhausts the memory.

We meet such condition in hour product environment as follow:
We have a table with five primary keys like k1, k2, ..., k5, all key types are UNSIGNED_LONG.
The query sql like " ... where k1 = 1 and k2 = 1 and k3 in (1,2,3,...,l) and k4 in (1,2,3,...,m) and k5 in (1,2,3,...,n)"
We have l=600, m=800 and n=1000, so the possible number of look up scans is 1*1*600*800*1000=480,000,000.
Each scan rowkey costs 5*8=40 bytes. Therefore the total memory cost is 480,000,000 * 50bytes = 25GB.
25GB exceeds the JMX configuration and causes OOM exception.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)