[jira] [Commented] (ATLAS-2774) Options for hard and soft delete of instances

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ATLAS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16687386#comment-16687386 ] 

ASF subversion and git services commented on ATLAS-2774:
--------------------------------------------------------

Commit d003ddb33b16a0a67e6b7f7eae29c1638a4d0971 in atlas's branch refs/heads/master from [~madhan@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=atlas.git;h=d003ddb ]

ATLAS-2774: enhancement to support parameterized delete-type (hard or soft) per API call


> Options for hard and soft delete of instances
> ---------------------------------------------
>
>                 Key: ATLAS-2774
>                 URL: https://issues.apache.org/jira/browse/ATLAS-2774
>             Project: Atlas
>          Issue Type: New Feature
>            Reporter: Graham Wallis
>            Assignee: Ashutosh Mestry
>            Priority: Major
>         Attachments: ATLAS-2774-Soft-and-hard-delete-impl.patch
>
>
> *Overview*
> For Entities and Relationships, the following delete behaviors are desired.
> Atlas can be configured to offer hard delete (only). Alternatively it can be configured to offer soft-delete. Configuration is achieved by setting the delete handler configuration property (see atlas-application.properties) to either the SoftDeleteHandlerV1 or the HardDeleteHandlerV1. The default (for when the property is not set) is to use the SoftDeleteHandlerV1.
> The AtlasEntityStoreV2 deleteById() and deleteByIds() and methods and the AtlasRelationshipStoreV2 deleteById() method are sensitive to the above configuration. When the configuration is soft these methods will perform a soft delete; when it is hard they will perform a hard delete.
> In addition to the above methods, the AtlasEntityStoreV2 and AtlasRelationshipStoreV2 offer a (new) purgeById() method that ALWAYS performs a hard delete. This is true regardless of which delete handler has been configured. When the configuration is hard, the purgeById() methods and deleteById() methods are essentially equivalent. When the configuration is soft, the purgeById() methods continue to provide a hard delete capability.
> The AtlasEntityStoreV2 deleteById(), deleteByIds() and purgeById() methods will delete the specified entities and any relationships to which they are connected. The AtlasRelationshipStoreV2 deleteById() and purgeById() methods will delete the specified relationship. Deletion of a relationship may cause an upate to an entity to which the relationhsip is connected, if it changes the propagation of classifications, for example.
> In a hard delete or purge operation, an affected entity and relationship will be removed from the graph and will not be returned in response to any future queries. This is true regardless of whether the affected entity or relationship has already been soft deleted or not. i.e. it's status could initially be ACTIVE or DELETED. Following the operation it will not exist.
> In a soft delete, the affected entities and relationships (that initially have status ACTIVE) are updated to set the status to DELETED. These instances can still be returned from queries and searches if the appropriate control is selected (to include deleted instances). Without that control selected they will not be returned.
> *Use Case for soft delete*
> Soft delete provides a 'safe' means of deleting instances from the repository whilst preserving an audit trail and (if supported by the repository) enabling a deleted instance to be restored (to ACTIVE status).
> *Use Cases for hard delete*
> One use case for hard delete is to permanently remove soft-deleted items some period of time after they were soft deleted.
>  Another use case for hard delete is to is to correct (clean up after) a mistake in which a potentially large batch of incomplete/corrupted/wrong metadata is loaded and needs to be fully removed.
> *Use Scenario for an OMAS user*
> An OMAS provides a delete function that does NOT offer the user a choice of hard/soft. The OMAS's delete function will invoke the relevant soft delete method provided by the OMRS - deleteEntity or deleteRelationship. These are both soft delete methods.
> The Atlas OMRS Connector's deleteEntity/deleteRelationship methods will check whether Atlas is configured for hard or soft delete. If Atlas is configured for hard delete then soft-delete is not possible, and the method will throw a FunctionNotSupported exception which is caught by the OMAS.
> On catching this exception the OMAS author should automatically escalate the soft delete to a hard delete by calling either of the mandatory OMRS purgeEntity() or purgeRelationship() methods. In the Atlas OMRS Connector these methods will invoke AtlasEntityStoreV2 purgeById() or AtlasRelationshipStoreV2 purgeById(). The effect of this is that if soft delete is not possible, a hard delete is used instead.
> An OMAS does NOT provide a purge capability to a non-privileged user, but a suitably expert OMRS or Atlas repository administrator can explicitly issue a purge call at the OMRS or Atlas interface, by calling purgeEntity() or purgeRelationship() (or the relevant Atlas store's purgeById() method).
> *Use Scenario for an Administrator*
> As highlighted above, a suitably expert repository Administrator can invoke the purgeEntity(), purgeRelationship() methods directly. They may need to do this to permanently remove instances following an earlier soft-delete and the timeout of a period of grace, or following a failed import or batch update.
> In addition to the above direct use by a repository administrator of the 'purge' methods, an Administrator could alternatively use the deleteEntity(), deleteRelationship() methods to perform soft deletes. The invoked method will behave as described above for an OMAS - performing a soft-delete if possible and throwing a FunctionNotSupported exception otherwise. In the latter case (no support for soft-delete), the Administrator COULD then choose to issue a purgeEntity(), purgeRelationship() call.
> *Restore of an entity or relationship*
> If an instance has been deleted using a soft-delete, the object still exists but has a status of 'DELETED'. Whilst in the DELETED state, the object can be:
>  * optionally included/excluded from search results
>  * restored using the OMRS method for restoreEntity or restoreRelationship.
>  * purged from the repository using the purgeEntity() or purgeRelationship() method (or the underlying Atlas store purgeById() method).
> The use case for restoring a soft-deleted entity or relationship is to correct an accidental deletion, or possibly a deliberately deletion that was performed for impact analysis.
> To restore an entity or relationship that has been soft-deleted, an OMAS user or repository administrator can use the restoreEntity() or restoreRealtionship() method of the OMRS connector.
> In the case of the Atlas OMRS Connector, the restoreEntity() and restoreRealtionship() methods will invoke (new) Atlas store methods to update the affected entities and relationships, leaving them in ACTIVE state and with classifications that reflect the propagation options between all the resulting active entities and relationships.
> *Within the Atlas Repository*
> The AtlasEntityStoreV2 and AtlasRelationshipStoreV2 provide additional methods for purgeById() that perform a hard delete, regardless of the choice of configured delete handler.
> The stores also provide additional methods to enable restore of an entity or relationship that has been soft-deleted.
> *Within the Atlas OMRS Connector*
> The Atlas OMRS Connector will interrogate the Atlas repository configuration to determine whether it is configuraed for soft or hard deletes. The Atlas OMRS Connector implementations of deleteEntity() and deleteRelationship() methods will therefore have prior knowledge as to whether it is feasible to attempt to perform a soft-delete. If feasible, the soft-delete will be issued to the relevant Atlas store. If not feasible, the methods will throw a FunctionNotSupported exception.
> Also within the Atlas OMRS Connector, the mandatory OMRS purgeEntity(), purgeRelationship() methods will invoke the Atlas stores' purgeById() methods to perform permanent (hard) removal of an entity or relationship from the repository. As outline above, these methods may be called either by an OMAS that has attempted a soft-delete and been 'bounced' or called directly by an Adminstrator.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)