[jira] [Resolved] (CARBONDATA-4162) Leverage Secondary Index till segment level with SI as datamap and SI with plan rewrite

["Kunal Kapoor (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CARBONDATA-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kunal Kapoor resolved CARBONDATA-4162.
--------------------------------------
    Fix Version/s: 2.2.0
       Resolution: Fixed

https://github.com/apache/carbondata/pull/4116

> Leverage Secondary Index till segment level with SI as datamap and SI with plan rewrite
> ---------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-4162
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4162
>             Project: CarbonData
>          Issue Type: New Feature
>            Reporter: Nihal kumar ojha
>            Priority: Major
>             Fix For: 2.2.0
>
>         Attachments: Support SI at segment level.pdf
>
>          Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> *Background:*
> Secondary index tables are created as indexes and managed as child tables internally by Carbondata. In the existing architecture, if the parent(main) table and SI table don’t
> have the same valid segments then we disable the SI table. And then from the
> next query onwards, we scan and prune only the parent table until we trigger
> the next load or REINDEX command (as these commands will make the
> parent and SI table segments in sync). Because of this, queries take more
> time to give the result when SI is disabled.
> *Proposed Solution:*
> We are planning to leverage SI till the segment level. It means at place
> of disabling the SI table(when parent and child table segments are not in sync)
> we will do pruning on SI tables for all the valid segments(segments with status
> success, marked for update and load partial success) and the rest of the
> segments will be pruned by the parent table.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)