You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Graham Wallis <gr...@uk.ibm.com> on 2018/07/02 07:50:34 UTC
Re: Deleting entities and relationships
Hi
ATLAS-2770 reminded me that this question is still pending. Would
appreciate responses/reactions as I'm trying to maximise the level of
function that can be supported by the Atlas OMRS connector - and at the
moment deletes are not looking great.
What I would like to implement is as follows. At the moment the clear
delineation in Atlas between configuration for soft-delete or hard-delete
makes some of the following impossible.
Allow the user to always be able to issue a soft-delete (as opposed to
only hard-deletes). This is a configuration choice - i.e. Atlas can
support this provided it is configured for soft-delete. However that
removes the option to purge/hard-delete which puts this in conflict with
the following requirement.
If the user has soft-deleted an entity or relationship, they should be
able to follow it up with a purge (hard delete) even though the repository
is configured for soft-delete.
If the user has soft-deleted an entity or relationship, they should be
able to undo that delete.
Prior to issuing a delete of an instance - the caller should be able to
inquire as to whether delete will be hard or soft using an API.
The caller should be able to determine from the result/response returned
by the delete method whether a delete was hard or soft.
Best regards,
Graham
Graham Wallis
IBM Analytics
Internet: graham_wallis@uk.ibm.com
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44 7741 379735
From: Graham Wallis/UK/IBM
To: "Madhan Neethiraj" <ma...@apache.org>
Cc: dev@atlas.apache.org
Date: 18/06/2018 16:44
Subject: Deleting entities and relationships
Hi Madhan
Could you please look at the following and tell me whether the following
look like sensible requirements for Atlas?
To support the OMRS interface, I would like to be able to:
soft delete an entity/relationship - this can fail if Atlas is configured
only for hard-deletes.
hard delete (purge) an entity/relationship - this is needed even when
Atlas is configured for soft deletes.
restore (un-delete) an entity/relationship that was soft-deleted
be able to purge (hard delete) an entity/relationship that was already
soft-deleted.
inquire whether a delete will be soft or hard (before issuing the delete)
- I guess being able to safely inspect the configuration would b an option
here, but an API might be better.
find out whether an entity/relationship was soft- or hard-deleted after
the delete operation. For this I could just try to retrieve the
entity/relationship but it would be neater to known from the return value.
I don't think the EntityMutationResponse includes that detail. I know the
requestContext is updated to include the deleted entities but that seems
to be before the delegation to either the soft or hard delete handler, so
I suspect it cannot indicate whether the delete was ultimately soft or
hard). I'm not sure how I would get that information that for
relationships.
Best regards,
Graham
Graham Wallis
IBM Analytics
Internet: graham_wallis@uk.ibm.com
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44 7741 379735
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: Deleting entities and relationships
Posted by Graham Wallis <gr...@uk.ibm.com>.
Thanks Madhan
Please see ATLAS-2774, in which I have outlined the use cases and possible
methods on the OMRS Connector and on the entity and relationship stores.
I do not know enough about the authorization model to comment on the purge
authority requirement - but it sounds necessary and I think is consistent
with the description in the above JIRA.
With regard to configuration I have coded up the metadatacollection to
inquire as to the delete handler configuration - I don't have an opinion
as to whether it should be available in the REST interface. I don't need
it to be reflected there. Others may want it for consistency.
Since the metadatacollection can now inquire the delete configuration I
think the requirement to report the outcome of a delete is no longer
needed - the connector will know in advance what outcome to expect (and
can verify it if needed).
Best regards,
Graham
Graham Wallis
IBM Analytics
Internet: graham_wallis@uk.ibm.com
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44 7741 379735
From: Madhan Neethiraj <ma...@apache.org>
To: Graham Wallis <gr...@uk.ibm.com>
Cc: "dev@atlas.apache.org" <de...@atlas.apache.org>
Date: 02/07/2018 23:22
Subject: Re: Deleting entities and relationships
Graham,
>> If the user has soft-deleted an entity or relationship, they should be
able to follow it up with a purge (hard delete) even though the repository
is configured for soft-delete
I agree. Having an API to purge soft-deleted entities/relationships will
be useful; criteria could be: 1) list-of-guids 2) time-since-delete.
Authorization model should be updated to require users to have explicit
permission to purge.
>> If the user has soft-deleted an entity or relationship, they should be
able to undo that delete
Can you give few usecases for this ask? Need to look into the details to
include in the notifications sent out by Atlas; today Atlas sends
ENTITY_CREATE/ENTITY_UPDATE/ENTITY_DELETE notifications.
>> Prior to issuing a delete of an instance - the caller should be able to
inquire as to whether delete will be hard or soft using an API
This is currently available at the server-side as configuration. Do you
think this info should be made available via REST?
>> The caller should be able to determine from the result/response
returned by the delete method whether a delete was hard or soft
Perhaps EntityMutations.EntityOperation can be updated with addition of
HARD_DELETE and SOFT_DELETE. Need to make sure to existing use of DELETE
continues, for backward compatibility with earlier clients.
Can you please file JIRA(s) to track these updates?
Thanks,
Madhan
From: Graham Wallis <gr...@uk.ibm.com>
Date: Monday, July 2, 2018 at 12:50 AM
To: Madhan Neethiraj <ma...@apache.org>
Cc: "dev@atlas.apache.org" <de...@atlas.apache.org>
Subject: Re: Deleting entities and relationships
Hi
ATLAS-2770 reminded me that this question is still pending. Would
appreciate responses/reactions as I'm trying to maximise the level of
function that can be supported by the Atlas OMRS connector - and at the
moment deletes are not looking great.
What I would like to implement is as follows. At the moment the clear
delineation in Atlas between configuration for soft-delete or hard-delete
makes some of the following impossible.
Allow the user to always be able to issue a soft-delete (as opposed to
only hard-deletes). This is a configuration choice - i.e. Atlas can
support this provided it is configured for soft-delete. However that
removes the option to purge/hard-delete which puts this in conflict with
the following requirement.
If the user has soft-deleted an entity or relationship, they should be
able to follow it up with a purge (hard delete) even though the repository
is configured for soft-delete.
If the user has soft-deleted an entity or relationship, they should be
able to undo that delete.
Prior to issuing a delete of an instance - the caller should be able to
inquire as to whether delete will be hard or soft using an API.
The caller should be able to determine from the result/response returned
by the delete method whether a delete was hard or soft.
Best regards,
Graham
Graham Wallis
IBM Analytics
Internet: graham_wallis@uk.ibm.com
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44 7741 379735
From: Graham Wallis/UK/IBM
To: "Madhan Neethiraj" <ma...@apache.org>
Cc: dev@atlas.apache.org
Date: 18/06/2018 16:44
Subject: Deleting entities and relationships
Hi Madhan
Could you please look at the following and tell me whether the following
look like sensible requirements for Atlas?
To support the OMRS interface, I would like to be able to:
soft delete an entity/relationship - this can fail if Atlas is configured
only for hard-deletes.
hard delete (purge) an entity/relationship - this is needed even when
Atlas is configured for soft deletes.
restore (un-delete) an entity/relationship that was soft-deleted
be able to purge (hard delete) an entity/relationship that was already
soft-deleted.
inquire whether a delete will be soft or hard (before issuing the delete)
- I guess being able to safely inspect the configuration would b an option
here, but an API might be better.
find out whether an entity/relationship was soft- or hard-deleted after
the delete operation. For this I could just try to retrieve the
entity/relationship but it would be neater to known from the return value.
I don't think the EntityMutationResponse includes that detail. I know the
requestContext is updated to include the deleted entities but that seems
to be before the delegation to either the soft or hard delete handler, so
I suspect it cannot indicate whether the delete was ultimately soft or
hard). I'm not sure how I would get that information that for
relationships.
Best regards,
Graham
Graham Wallis
IBM Analytics
Internet: graham_wallis@uk.ibm.com
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44 7741 379735
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Re: Deleting entities and relationships
Posted by Madhan Neethiraj <ma...@apache.org>.
Graham,
>> If the user has soft-deleted an entity or relationship, they should be able to follow it up with a purge (hard delete) even though the repository is configured for soft-delete
I agree. Having an API to purge soft-deleted entities/relationships will be useful; criteria could be: 1) list-of-guids 2) time-since-delete. Authorization model should be updated to require users to have explicit permission to purge.
>> If the user has soft-deleted an entity or relationship, they should be able to undo that delete
Can you give few usecases for this ask? Need to look into the details to include in the notifications sent out by Atlas; today Atlas sends ENTITY_CREATE/ENTITY_UPDATE/ENTITY_DELETE notifications.
>> Prior to issuing a delete of an instance - the caller should be able to inquire as to whether delete will be hard or soft using an API
This is currently available at the server-side as configuration. Do you think this info should be made available via REST?
>> The caller should be able to determine from the result/response returned by the delete method whether a delete was hard or soft
Perhaps EntityMutations.EntityOperation can be updated with addition of HARD_DELETE and SOFT_DELETE. Need to make sure to existing use of DELETE continues, for backward compatibility with earlier clients.
Can you please file JIRA(s) to track these updates?
Thanks,
Madhan
From: Graham Wallis <gr...@uk.ibm.com>
Date: Monday, July 2, 2018 at 12:50 AM
To: Madhan Neethiraj <ma...@apache.org>
Cc: "dev@atlas.apache.org" <de...@atlas.apache.org>
Subject: Re: Deleting entities and relationships
Hi
ATLAS-2770 reminded me that this question is still pending. Would appreciate responses/reactions as I'm trying to maximise the level of function that can be supported by the Atlas OMRS connector - and at the moment deletes are not looking great.
What I would like to implement is as follows. At the moment the clear delineation in Atlas between configuration for soft-delete or hard-delete makes some of the following impossible.
Allow the user to always be able to issue a soft-delete (as opposed to only hard-deletes). This is a configuration choice - i.e. Atlas can support this provided it is configured for soft-delete. However that removes the option to purge/hard-delete which puts this in conflict with the following requirement.
If the user has soft-deleted an entity or relationship, they should be able to follow it up with a purge (hard delete) even though the repository is configured for soft-delete.
If the user has soft-deleted an entity or relationship, they should be able to undo that delete.
Prior to issuing a delete of an instance - the caller should be able to inquire as to whether delete will be hard or soft using an API.
The caller should be able to determine from the result/response returned by the delete method whether a delete was hard or soft.
Best regards,
Graham
Graham Wallis
IBM Analytics
Internet: graham_wallis@uk.ibm.com
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44 7741 379735
From: Graham Wallis/UK/IBM
To: "Madhan Neethiraj" <ma...@apache.org>
Cc: dev@atlas.apache.org
Date: 18/06/2018 16:44
Subject: Deleting entities and relationships
Hi Madhan
Could you please look at the following and tell me whether the following look like sensible requirements for Atlas?
To support the OMRS interface, I would like to be able to:
soft delete an entity/relationship - this can fail if Atlas is configured only for hard-deletes.
hard delete (purge) an entity/relationship - this is needed even when Atlas is configured for soft deletes.
restore (un-delete) an entity/relationship that was soft-deleted
be able to purge (hard delete) an entity/relationship that was already soft-deleted.
inquire whether a delete will be soft or hard (before issuing the delete) - I guess being able to safely inspect the configuration would b an option here, but an API might be better.
find out whether an entity/relationship was soft- or hard-deleted after the delete operation. For this I could just try to retrieve the entity/relationship but it would be neater to known from the return value. I don't think the EntityMutationResponse includes that detail. I know the requestContext is updated to include the deleted entities but that seems to be before the delegation to either the soft or hard delete handler, so I suspect it cannot indicate whether the delete was ultimately soft or hard). I'm not sure how I would get that information that for relationships.
Best regards,
Graham
Graham Wallis
IBM Analytics
Internet: graham_wallis@uk.ibm.com
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44 7741 379735
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU