[jira] [Comment Edited] (ATLAS-541) Soft deletes

["Shwetha G S (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ATLAS-541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223653#comment-15223653 ] 

Shwetha G S edited comment on ATLAS-541 at 4/4/16 4:36 AM:
-----------------------------------------------------------

Here is the proposed approach:
1. Introduce entity state - ACTIVE/DELETED. The status should be part of Id json object(with guid, version, typename) so that the clients can use this information.
2. At entity create, check if there is an entity with same unique attribute and state = ACTIVE
  a. If there is another entity, do not create new entity
  b. If there is no other entity, create new entity and mark the state as ACTIVE
3. At entity delete, instead of deleting the entity, mark the entity and its related entities(composite entities) as DELETED
4. In search, when the entity is returned, there should be entity state as well in the results. 

Not in scope:
Ideally, search by default should be just on ACTIVE entities. Also, there should be an option to search on DELETED entities as well. However, search will be done in the 2nd phase. In the first phase, the clients will have to filter the results depending on their requirements.The cavaet here is, if the search doesn't return back the entity, just subset of the columns(like DSL query - hive_db select name where only name attribute is returned), entity state is not returned in the results and hence there is no way to filter the results based on entity state on the client side. Also, entity lineage will not return entity state.


was (Author: shwethags):
Here is the proposed approach:
1. Introduce entity state - ACTIVE/DELETED. The status should be part of Id json object(with guid, version, typename) so that the clients can use this information.
2. At entity create, check if there is an entity with same unique attribute and state = ACTIVE
  a. If there is another entity, do not create new entity
  b. If there is no other entity, create new entity and mark the state as ACTIVE
3. At entity delete, instead of deleting the entity, mark the entity and its related entities(composite entities) as DELETED
4. In search, when the entity is returned, there should be entity state as well in the results. 

Not in scope:
Ideally, search by default should be just on ACTIVE entities. But there should be an option to search in DELETED entities as well. However, search will be done in the 2nd phase. In the first phase, the clients have to filter the results depending on their requirements.The cavaet here is, if the search doesn't return back the entity, just subset of the columns(like DSL query - hive_db select name), entity state is not returned in the results and hence there is no way to filter the results based on entity state on the client side. Also, entity lineage will not return entity state.

> Soft deletes
> ------------
>
>                 Key: ATLAS-541
>                 URL: https://issues.apache.org/jira/browse/ATLAS-541
>             Project: Atlas
>          Issue Type: New Feature
>            Reporter: Shwetha G S
>            Assignee: Shwetha G S
>
> We don't have graph versioning currently and hard deletes are not acceptable for data governance. This jira tracks the proposal for soft deletes which can mark an entity as deleted and by default search should return only active entities. However, there should be an option to retrieve deleted entities



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)