You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@atlas.apache.org by "Jagadesh Kiran N (Jira)" <ji...@apache.org> on 2024/10/01 17:31:00 UTC

[jira] [Created] (ATLAS-4909) Pruning of old updates of an entity in apache atlas

Jagadesh Kiran N created ATLAS-4909:
---------------------------------------

             Summary:  Pruning of old updates of an entity  in apache atlas 
                 Key: ATLAS-4909
                 URL: https://issues.apache.org/jira/browse/ATLAS-4909
             Project: Atlas
          Issue Type: Bug
    Affects Versions: 2.3.0
            Reporter: Jagadesh Kiran N
         Attachments: image.png

Use case : Updating same entity multiple times ,  Atlas is creating multiple new records for every update in Hbase janus table .

This resulted in  increase in records &  storage size and if it continues to scale up,  disk space also exponentially increase .

If we want to prune or purge old updates of the entity , atlas doesn't have any API or configuration to do so. 

But at one point of time this will break when disk size exhausts.  

So there should be a way to delete old updates of entity and keep the latest one or  last three versions of updates . 

Attached screenshot with sample computation executed with atlas for scale and its impact on Storage ( HDFS Size) . 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)