You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Paresh Devalia <pa...@freestoneinfotech.com> on 2022/05/05 05:28:52 UTC
Review Request 73974: ATLAS-25071 Deferred Actions : Support to add classification name instead of classificationVertexId
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73974/
-----------------------------------------------------------
Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and Sharmadha Sainath.
Bugs: ATLAS-25071
https://issues.apache.org/jira/browse/ATLAS-25071
Repository: atlas
Description
-------
In /api/atlas/admin/tasks call , when a tag is propagated , the classificationVertexId is displayed. It becomes tough to debug when there are many pending tasks . classification name can be added in the response in addition to/instead of classificationVertexId.
Diffs
-----
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f118ae69a
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 9a7f29017
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java 0bad84e49
Diff: https://reviews.apache.org/r/73974/diff/1/
Testing
-------
Manual testing is done.
Pre-commit : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1091/consoleFull
Created 1000 hive_tables dependant on parent table.
Hive_tables name will be sequence_1 to sequence_999
Below step are tested for add, update and delete classification.
1) Adding classification :-
Create a classification with name test_1.
Added test_1 classification with checked on "propagate" of sequence_1 hive_table, so this classification will be assign to all child table.
So while background process is going on, will check the task status API.
curl k -u <userid>:<password> -request GET 'https://<hostname>:<port>/api/atlas/admin/tasks'
so in response will be
[
{
"type": "CLASSIFICATION_PROPAGATION_ADD",
"guid": "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
"createdBy": "admin",
"createdTime": 1651591949394,
"updatedTime": 1651591949394,
"parameters": {
"relationshipGuid": null,
"classificationName": "test_1",
"entityGuid": "29a5ac89-a3fb-46ff-bbb1-646d3b0717dd",
"classificationVertexId": "122896440"
},
"attemptCount": 0,
"status": "PENDING"
}
]
2)Updating classification attribute value:-
Creating attribute for test_1 classification.
Updating classification attribute value on sequence_1 hive_table, then in background process it will update all the child table attribute value.
So while background process is going on, will check the task status API.
curl k -u <userid>:<password> -request GET 'https://<hostname>:<port>/api/atlas/admin/tasks'
so in response will be
[
{
"type": "CLASSIFICATION_PROPAGATION_ADD",
"guid": "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
"createdBy": "admin",
"createdTime": 1651591949394,
"updatedTime": 1651591949394,
"parameters": {
"relationshipGuid": null,
"classificationName": "test_1",
"entityGuid": "29a5ac89-a3fb-46ff-bbb1-646d3b0717dd",
"classificationVertexId": "122896440"
},
"attemptCount": 0,
"status": "PENDING"
}
]
3)Deleteing classification:-
Create a classification with name test_2.
Added test_1 classification with checked on "propagte" and "Remove propagation on entity delete" on sequence_1 hive_table, so this classification will be assign to
all child table.
So background process is going on, will check the task status API.
curl k -u <userid>:<password> -request GET 'https://<hostname>:<port>/api/atlas/admin/tasks'
so in response will be
[
{
"type": "CLASSIFICATION_PROPAGATION_ADD",
"guid": "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
"createdBy": "admin",
"createdTime": 1651591949394,
"updatedTime": 1651591949394,
"parameters": {
"relationshipGuid": null,
"classificationName": "test_2",
"entityGuid": "29a5ac89-a3fb-46ff-bbb1-646d3b0717dd",
"classificationVertexId": "122896440"
},
"attemptCount": 0,
"status": "PENDING"
}
]
Will run an delete API and provide a guid of hive_table which want to delete.
curl k -u <userid>:<password> -request DELETE 'https://<hostname>:<port>api/atlas/v2/entity/bulk?guid={guid}'
So while delete process is going on, will check the task status API.
curl k -u <userid>:<password> -request GET 'https://<hostname>:<port>/api/atlas/admin/tasks'
so in response will be
[
{
"type": "CLASSIFICATION_PROPAGATION_DELETE",
"guid": "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
"createdBy": "admin",
"createdTime": 1651591949394,
"updatedTime": 1651591949394,
"parameters": {
"relationshipGuid": null,
"classificationName": "test_2",
"entityGuid": "29a5ac89-a3fb-46ff-bbb1-646d3b0717dd",
"classificationVertexId": "122896440"
},
"attemptCount": 0,
"status": "PENDING"
}
]
Thanks,
Paresh Devalia
Re: Review Request 73974: ATLAS-25071 Deferred Actions : Support to add classification name instead of classificationVertexId
Posted by Paresh Devalia <pa...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73974/
-----------------------------------------------------------
(Updated May 9, 2022, 6:08 a.m.)
Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and Sharmadha Sainath.
Bugs: ATLAS-25071
https://issues.apache.org/jira/browse/ATLAS-25071
Repository: atlas
Description
-------
In /api/atlas/admin/tasks call , when a tag is propagated , the classificationVertexId is displayed. It becomes tough to debug when there are many pending tasks . classification name can be added in the response in addition to/instead of classificationVertexId.
Diffs
-----
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f118ae69a
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 9a7f29017
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java 0bad84e49
Diff: https://reviews.apache.org/r/73974/diff/1/
Testing (updated)
-------
Manual testing is done.
Pre-commit : https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1095/consoleFull
Created 1000 hive_tables dependant on parent table.
Hive_tables name will be sequence_1 to sequence_999
Below step are tested for add, update and delete classification.
1) Adding classification :-
Create a classification with name test_1.
Added test_1 classification with checked on "propagate" of sequence_1 hive_table, so this classification will be assign to all child table.
So while background process is going on, will check the task status API.
curl k -u <userid>:<password> -request GET 'https://<hostname>:<port>/api/atlas/admin/tasks'
so in response will be
[
{
"type": "CLASSIFICATION_PROPAGATION_ADD",
"guid": "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
"createdBy": "admin",
"createdTime": 1651591949394,
"updatedTime": 1651591949394,
"parameters": {
"relationshipGuid": null,
"classificationName": "test_1",
"entityGuid": "29a5ac89-a3fb-46ff-bbb1-646d3b0717dd",
"classificationVertexId": "122896440"
},
"attemptCount": 0,
"status": "PENDING"
}
]
2)Updating classification attribute value:-
Creating attribute for test_1 classification.
Updating classification attribute value on sequence_1 hive_table, then in background process it will update all the child table attribute value.
So while background process is going on, will check the task status API.
curl k -u <userid>:<password> -request GET 'https://<hostname>:<port>/api/atlas/admin/tasks'
so in response will be
[
{
"type": "CLASSIFICATION_PROPAGATION_ADD",
"guid": "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
"createdBy": "admin",
"createdTime": 1651591949394,
"updatedTime": 1651591949394,
"parameters": {
"relationshipGuid": null,
"classificationName": "test_1",
"entityGuid": "29a5ac89-a3fb-46ff-bbb1-646d3b0717dd",
"classificationVertexId": "122896440"
},
"attemptCount": 0,
"status": "PENDING"
}
]
3)Deleteing classification:-
Create a classification with name test_2.
Added test_1 classification with checked on "propagte" and "Remove propagation on entity delete" on sequence_1 hive_table, so this classification will be assign to
all child table.
So background process is going on, will check the task status API.
curl k -u <userid>:<password> -request GET 'https://<hostname>:<port>/api/atlas/admin/tasks'
so in response will be
[
{
"type": "CLASSIFICATION_PROPAGATION_ADD",
"guid": "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
"createdBy": "admin",
"createdTime": 1651591949394,
"updatedTime": 1651591949394,
"parameters": {
"relationshipGuid": null,
"classificationName": "test_2",
"entityGuid": "29a5ac89-a3fb-46ff-bbb1-646d3b0717dd",
"classificationVertexId": "122896440"
},
"attemptCount": 0,
"status": "PENDING"
}
]
Will run an delete API and provide a guid of hive_table which want to delete.
curl k -u <userid>:<password> -request DELETE 'https://<hostname>:<port>api/atlas/v2/entity/bulk?guid={guid}'
So while delete process is going on, will check the task status API.
curl k -u <userid>:<password> -request GET 'https://<hostname>:<port>/api/atlas/admin/tasks'
so in response will be
[
{
"type": "CLASSIFICATION_PROPAGATION_DELETE",
"guid": "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
"createdBy": "admin",
"createdTime": 1651591949394,
"updatedTime": 1651591949394,
"parameters": {
"relationshipGuid": null,
"classificationName": "test_2",
"entityGuid": "29a5ac89-a3fb-46ff-bbb1-646d3b0717dd",
"classificationVertexId": "122896440"
},
"attemptCount": 0,
"status": "PENDING"
}
]
Thanks,
Paresh Devalia