[jira] [Created] (CARBONDATA-2092) Fix compaction bug to prevent the compaction flow from going through the restructure compaction flow

["Manish Gupta (JIRA)" <ji...@apache.org>]

Manish Gupta created CARBONDATA-2092:
----------------------------------------

             Summary: Fix compaction bug to prevent the compaction flow from going through the restructure compaction flow
                 Key: CARBONDATA-2092
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2092
             Project: CarbonData
          Issue Type: Bug
            Reporter: Manish Gupta
            Assignee: Manish Gupta


Problem and analysis:

----------------------------------------

During data load current schema timestamp is written to the carbondata fileHeader. This is used during compaction to decide whether the block is a restructured block or the block is according to the latest schema.

As the blocklet information is now stored in the index file, while laoding it in memory the carbondata file header is not read and due to this the schema timestamp is not getting set to the blocklet information. Due to this during compaction flow there is a mismatch on comparing the current schema time stamp with the timestamp stored in the block and the flow goes through the restructure compaction flow instead of normal compaction flow.

Impact:

-------------

Compaction performance degradation as restructure compaction flow involves sorting of data again.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)