You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@carbondata.apache.org by "Xingjun Hao (Jira)" <ji...@apache.org> on 2020/12/13 19:22:00 UTC

[jira] [Created] (CARBONDATA-4083) Refactor Update and Support Update Atomicity

Xingjun Hao created CARBONDATA-4083:
---------------------------------------

             Summary: Refactor Update and Support Update Atomicity
                 Key: CARBONDATA-4083
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4083
             Project: CarbonData
          Issue Type: Improvement
            Reporter: Xingjun Hao


Currently, we will modify tablestatus file for serveral times in the update flow. In total 4 tablestauts write ops destoy the Atomicity to a certain extent. which maybe incur dirty data under update failure scenrios.

The first time we update tablestatus is when writing delta files, firstly we update the updatedeltastarttime and updatedeltaendtime in the tablestatus, then delete some segments, which bring 2 tablestatus write ops.



The second time we update tatblstatus is when insert new data. just like the first time, will bring 2 tablesatus write ops.

Also, auto compaction doesn't work for UPDATE. UPDATE won't trigger MINOR Compaction even when we TURN ON carbon.merge.auto.compaction.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)