You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@carbondata.apache.org by "Ravindra Pesala (JIRA)" <ji...@apache.org> on 2017/06/27 06:29:00 UTC

[jira] [Created] (CARBONDATA-1230) Datamap framework for Carbondata to leverage indexing

Ravindra Pesala created CARBONDATA-1230:
-------------------------------------------

             Summary: Datamap framework for Carbondata to leverage indexing
                 Key: CARBONDATA-1230
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1230
             Project: CarbonData
          Issue Type: New Feature
            Reporter: Ravindra Pesala


Datamap should be single point interface for indexing and pruning. 
It could be two types
# 1. Coarse grained datamap.
# 2 Fine grained datamap.

h3. Coarse grained datamap
These datamaps contains the information of blocklets. so it can prune till blocklet level. It could be loaded on driver side or executor side depends on size of datamap.
Default implementation for this type is BlockletDataMap. It contains all necessary information  of blocklet with stats like startkey, endkey and max and min value. Using this information all filter queries would be pruned by datamap.

h3. Fine grained datamap
These datamap contains information up to page and row level. It is stored executor side and used as part of filtering to speed up the queries.
 




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)