[jira] [Created] (CARBONDATA-3745) Support column chunk data cache

["Jacky Li (Jira)" <ji...@apache.org>]

Jacky Li created CARBONDATA-3745:
------------------------------------

             Summary: Support column chunk data cache
                 Key: CARBONDATA-3745
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3745
             Project: CarbonData
          Issue Type: New Feature
            Reporter: Jacky Li
            Assignee: Jacky Li


In some environment where memory is more but IO is slow, cache is a good technology to accelerate the query performance.

There are many existing cache solution that carbon can leverage, like Alluxio, Redis. However, these solutions are caching block level object which may require big memory in some case. 

CarbonData is a columnar file format, and IO unit is in column chunk wise, so it is preferred to support column chunk level caching instead of whole file block. In this way, cache requires less memory thus we can cache more data in memory, so hopefully we can get higher cache hit rate. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)