You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Pinal Shah <pi...@freestoneinfotech.com> on 2024/10/04 04:37:19 UTC
Review Request 75225: ATLAS-4908: Handle deferred action pending tasks, once the feature is disabled and enabled.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75225/
-----------------------------------------------------------
Review request for atlas, Jayendra Parab, Radhika Kundam, and Sarath Subramanian.
Bugs: ATLAS-4908
https://issues.apache.org/jira/browse/ATLAS-4908
Repository: atlas
Description
-------
**Scenario:**
Suppose Atlas has many pending task in the background
If we disable atlas.tasks.enable=false, i.e disable deferred action, then we can loose all the pending tasks related to classifications add or delete.
Plan was to create a patch to run all the pending tasks, but if the pending task are large in number, then it can take much time for Atlas startup.
**Use cases:**
CLASSIFICATION_PROPAGATION_ADD
Classification applied on parent has been removed
New attribute has been added in Classification which is applied on parent
classification has been removed, applied again applied
Entity has been soft deleted with remove propagated tags as 'false'
Entity has been soft deleted with remove propagated tags as 'true'
CLASSIFICATION_PROPAGATION_DELETE
Same classification applied on the parent
Same classification applied on the parent and then removed
Entity has been soft deleted
Diffs
-----
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 6828d0c2f
Diff: https://reviews.apache.org/r/75225/diff/1/
Testing
-------
Tested manually
Thanks,
Pinal Shah
Re: Review Request 75225: ATLAS-4908: Handle deferred action pending tasks, once the feature is disabled and enabled.
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75225/#review226969
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Oct. 8, 2024, 3:13 a.m., Pinal Shah wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/75225/
> -----------------------------------------------------------
>
> (Updated Oct. 8, 2024, 3:13 a.m.)
>
>
> Review request for atlas, Jayendra Parab, Radhika Kundam, and Sarath Subramanian.
>
>
> Bugs: ATLAS-4908
> https://issues.apache.org/jira/browse/ATLAS-4908
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Scenario:**
>
> If we disable atlas.tasks.enable=false, i.e disable deferred action, do some operation related to classification on same entity which is present in pending task and enable deferred action back, this might cause some inconsistency.
>
> **Below are the operation that might get performed on the entity while deferred action is disabled, and there are pending task present related to same entity:**
>
> Task Type : CLASSIFICATION_PROPAGATION_ADD
>
> - Classification applied on parent has been removed
> - New attribute has been added in Classification which is applied on parent
> - classification has been removed, applied again applied
> - Entity has been soft deleted with remove propagated tags as 'false'
> - Entity has been soft deleted with remove propagated tags as 'true'
>
> Task Type : CLASSIFICATION_PROPAGATION_DELETE
>
> - Same classification applied on the parent
> - Same classification applied on the parent and then removed
> - Entity has been soft deleted
>
> Task Type : CLASSIFICATION_PROPAGATION_RELATIONSHIP_UPDATE
>
> - if PropagateTag value is updated
> - if blockedClassification list has been updated (not handled in this patch)
>
> TODO: add in documentation, the consequences of disabling deferred action like Audit Reduction/Ageing impact, etc.
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 6828d0c2f
>
>
> Diff: https://reviews.apache.org/r/75225/diff/2/
>
>
> Testing
> -------
>
> Tested manually
>
>
> Thanks,
>
> Pinal Shah
>
>
Re: Review Request 75225: ATLAS-4908: Handle deferred action pending tasks, once the feature is disabled and enabled.
Posted by Pinal Shah <pi...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/75225/
-----------------------------------------------------------
(Updated Oct. 8, 2024, 3:13 a.m.)
Review request for atlas, Jayendra Parab, Radhika Kundam, and Sarath Subramanian.
Changes
-------
updated description and minor change
Bugs: ATLAS-4908
https://issues.apache.org/jira/browse/ATLAS-4908
Repository: atlas
Description (updated)
-------
**Scenario:**
If we disable atlas.tasks.enable=false, i.e disable deferred action, do some operation related to classification on same entity which is present in pending task and enable deferred action back, this might cause some inconsistency.
**Below are the operation that might get performed on the entity while deferred action is disabled, and there are pending task present related to same entity:**
Task Type : CLASSIFICATION_PROPAGATION_ADD
- Classification applied on parent has been removed
- New attribute has been added in Classification which is applied on parent
- classification has been removed, applied again applied
- Entity has been soft deleted with remove propagated tags as 'false'
- Entity has been soft deleted with remove propagated tags as 'true'
Task Type : CLASSIFICATION_PROPAGATION_DELETE
- Same classification applied on the parent
- Same classification applied on the parent and then removed
- Entity has been soft deleted
Task Type : CLASSIFICATION_PROPAGATION_RELATIONSHIP_UPDATE
- if PropagateTag value is updated
- if blockedClassification list has been updated (not handled in this patch)
TODO: add in documentation, the consequences of disabling deferred action like Audit Reduction/Ageing impact, etc.
Diffs (updated)
-----
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 6828d0c2f
Diff: https://reviews.apache.org/r/75225/diff/2/
Changes: https://reviews.apache.org/r/75225/diff/1-2/
Testing
-------
Tested manually
Thanks,
Pinal Shah