You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Sidharth Mishra <si...@gmail.com> on 2020/03/30 19:03:30 UTC
Review Request 72284: ATLAS-3707: Atlas Purge entity is generating
both purge and delete audit
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72284/
-----------------------------------------------------------
Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian.
Bugs: ATLAS-3707
https://issues.apache.org/jira/browse/ATLAS-3707
Repository: atlas
Description
-------
The check is been added, so that when its purge operation there will not be any entity delete notification sent to audit
Diffs
-----
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 75b016cca
Diff: https://reviews.apache.org/r/72284/diff/1/
Testing
-------
Manual testing.
curl --location --request GET 'http://sid-cdp-3-1.gce.cloudera.com:31000/api/atlas/v2/entity/d7d29f3e-1ef6-4baa-b66c-3f20d2839036/audit' \
--header 'Authorization: Basic YWRtaW46YWRtaW4='
Thanks,
Sidharth Mishra
Re: Review Request 72284: ATLAS-3707: Atlas Purge entity is
generating both purge and delete audit
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72284/#review220285
-----------------------------------------------------------
Ship it!
Ship It!
- Sarath Subramanian
On March 30, 2020, 12:03 p.m., Sidharth Mishra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72284/
> -----------------------------------------------------------
>
> (Updated March 30, 2020, 12:03 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3707
> https://issues.apache.org/jira/browse/ATLAS-3707
>
>
> Repository: atlas
>
>
> Description
> -------
>
> The check is been added, so that when its purge operation there will not be any entity delete notification sent to audit
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 75b016cca
>
>
> Diff: https://reviews.apache.org/r/72284/diff/1/
>
>
> Testing
> -------
>
> Manual testing.
>
> curl --location --request GET 'http://sid-cdp-3-1.gce.cloudera.com:31000/api/atlas/v2/entity/d7d29f3e-1ef6-4baa-b66c-3f20d2839036/audit' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4='
>
>
> Thanks,
>
> Sidharth Mishra
>
>
Re: Review Request 72284: ATLAS-3707: Atlas Purge entity is
generating both purge and delete audit
Posted by Sidharth Mishra <si...@gmail.com>.
> On March 30, 2020, 7:25 p.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
> > Lines 325 (patched)
> > <https://reviews.apache.org/r/72284/diff/1/?file=2216465#file2216465line325>
> >
> > In case of purge, shouldn't these entities be not added to resp as PURGEed entities? Like:
> >
> > EntityOperation deleteOperation = req.isPurgeRequested() ? PURGE : DELETE;
> >
> > for (AtlasEntityHeader entity : req.getDeletedEntities()) {
> > resp.addEntity(deleteOperation, entity);
> > }
> >
> > Alternatively, consider adding RequestContext.purgedEntities and add to this collection (instead of deletedEntities) while performing purge - from DeleteHandlerV1.deleteEntities():
> >
> > for (GraphHelper.VertexInfo vertexInfo : getOwnedVertices(instanceVertex)) {
> > if (requestContext.isPurgeRequested()) {
> > requestContext.recordEntityPurge(vertexInfo.getEntity());
> > } else {
> > requestContext.recordEntityDelete(vertexInfo.getEntity());
> > }
> >
> > deletionCandidateVertices.add(vertexInfo.getVertex());
> > }
At org/apache/atlas/web/resources/AdminResource.java:473, EntityMutationResponse resp = entityStore.purgeByIds(guids); keeps the deleted entities in requestContext and we get the same and assign it to response as you correctly pointed above.
After that we again create atlas audit entity at org/apache/atlas/web/resources/AdminResource.java:480 which eventually calls org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2#createOrUpdate(org.apache.atlas.repository.store.graph.v2.EntityStream, boolean, boolean, boolean) where we see if there are deleted entities at request context then we generate the audit for those which we need to stop by adding the check. To keep the hard delete and purge code same we are still using the requestContext deletedEntities instead of having separate one for purge.
- Sidharth
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72284/#review220121
-----------------------------------------------------------
On March 30, 2020, 7:03 p.m., Sidharth Mishra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72284/
> -----------------------------------------------------------
>
> (Updated March 30, 2020, 7:03 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3707
> https://issues.apache.org/jira/browse/ATLAS-3707
>
>
> Repository: atlas
>
>
> Description
> -------
>
> The check is been added, so that when its purge operation there will not be any entity delete notification sent to audit
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 75b016cca
>
>
> Diff: https://reviews.apache.org/r/72284/diff/1/
>
>
> Testing
> -------
>
> Manual testing.
>
> curl --location --request GET 'http://sid-cdp-3-1.gce.cloudera.com:31000/api/atlas/v2/entity/d7d29f3e-1ef6-4baa-b66c-3f20d2839036/audit' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4='
>
>
> Thanks,
>
> Sidharth Mishra
>
>
Re: Review Request 72284: ATLAS-3707: Atlas Purge entity is
generating both purge and delete audit
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72284/#review220121
-----------------------------------------------------------
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
Lines 325 (patched)
<https://reviews.apache.org/r/72284/#comment308449>
In case of purge, shouldn't these entities be not added to resp as PURGEed entities? Like:
EntityOperation deleteOperation = req.isPurgeRequested() ? PURGE : DELETE;
for (AtlasEntityHeader entity : req.getDeletedEntities()) {
resp.addEntity(deleteOperation, entity);
}
Alternatively, consider adding RequestContext.purgedEntities and add to this collection (instead of deletedEntities) while performing purge - from DeleteHandlerV1.deleteEntities():
for (GraphHelper.VertexInfo vertexInfo : getOwnedVertices(instanceVertex)) {
if (requestContext.isPurgeRequested()) {
requestContext.recordEntityPurge(vertexInfo.getEntity());
} else {
requestContext.recordEntityDelete(vertexInfo.getEntity());
}
deletionCandidateVertices.add(vertexInfo.getVertex());
}
- Madhan Neethiraj
On March 30, 2020, 7:03 p.m., Sidharth Mishra wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72284/
> -----------------------------------------------------------
>
> (Updated March 30, 2020, 7:03 p.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3707
> https://issues.apache.org/jira/browse/ATLAS-3707
>
>
> Repository: atlas
>
>
> Description
> -------
>
> The check is been added, so that when its purge operation there will not be any entity delete notification sent to audit
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 75b016cca
>
>
> Diff: https://reviews.apache.org/r/72284/diff/1/
>
>
> Testing
> -------
>
> Manual testing.
>
> curl --location --request GET 'http://sid-cdp-3-1.gce.cloudera.com:31000/api/atlas/v2/entity/d7d29f3e-1ef6-4baa-b66c-3f20d2839036/audit' \
> --header 'Authorization: Basic YWRtaW46YWRtaW4='
>
>
> Thanks,
>
> Sidharth Mishra
>
>