You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ashutosh Mestry via Review Board <no...@reviews.apache.org> on 2019/10/26 05:23:11 UTC
Review Request 71683: ATLAS-3489: Import API: missing entity in atlas
after import
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71683/
-----------------------------------------------------------
Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian.
Bugs: ATLAS-3489
https://issues.apache.org/jira/browse/ATLAS-3489
Repository: atlas
Description
-------
Bug Details:
Input :
* Consider case where input zip contains hive_table which has 2 hive_columns with same qualifiedName but different entityStatus.
* One of the columns is ACTIVE while other is DELETED.
Expected Result :
Imported table must contain 2 columns with same qualifiedName but with different entityStatus.
Actual Result :
Imported table has 1 column with guid of ACTIVE column from input zip & entityStatus as DELETED.
Problem Area :
* Problem is in AtlasEntityStoreV2.preCreateOrUpdate().getResolvedEntityVertex(discoveryContext, entity), #951.
* This method fetches already resolved vertex for column with same qualifiedName since both entities have same uniqueAttributes, where as it should return null vertex & further new vertex should be created for entity with different entityStatus.
Solution :
* Moved field 'entityStatus' from class AtlasRelatedObjectId to class AtlasObjectId.
* Changed constructor of AtlasObjectId used in EntityGraphRetriever.toAtlasObjectId(AtlasEntity entity).
Diffs
-----
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java 5b3c27dff
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java 95f6ec3bb
Diff: https://reviews.apache.org/r/71683/diff/1/
Testing
-------
Added new UT -> ImportServiceTest.testImportDuplicateColumnsWithDifferentStatus
Verified all existing UTs locally with the patch.
Tested manually.
Thanks,
Ashutosh Mestry
Re: Review Request 71683: ATLAS-3489: Import API: missing entity in
atlas after import
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71683/#review218415
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Oct. 27, 2019, 5:20 p.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71683/
> -----------------------------------------------------------
>
> (Updated Oct. 27, 2019, 5:20 p.m.)
>
>
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3489
> https://issues.apache.org/jira/browse/ATLAS-3489
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Bug Details:
>
> Input :
> * Consider case where input zip contains hive_table which has 2 hive_columns with same qualifiedName but different entityStatus.
> * One of the columns is ACTIVE while other is DELETED.
>
> Expected Result :
>
> Imported table must contain 2 columns with same qualifiedName but with different entityStatus.
>
> Actual Result :
>
> Imported table has 1 column with guid of ACTIVE column from input zip & entityStatus as DELETED.
>
>
> Problem Area :
>
> * Problem is in AtlasEntityStoreV2.preCreateOrUpdate().getResolvedEntityVertex(discoveryContext, entity), #951.
> * This method fetches already resolved vertex for column with same qualifiedName since both entities have same uniqueAttributes, where as it should return null vertex & further new vertex should be created for entity with different entityStatus.
>
> Solution :
>
> * Added condition to retain entity's GUID when import is in progress.
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java 5b3c27dff
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java 95f6ec3bb
>
>
> Diff: https://reviews.apache.org/r/71683/diff/1/
>
>
> Testing
> -------
>
> Added new UT -> ImportServiceTest.testImportDuplicateColumnsWithDifferentStatus
> Verified all existing UTs locally with the patch.
> Tested manually.
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 71683: ATLAS-3489: Import API: missing entity in
atlas after import
Posted by Ashutosh Mestry via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71683/
-----------------------------------------------------------
(Updated Oct. 27, 2019, 5:20 p.m.)
Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian.
Changes
-------
Updates include: Solution section.
Bugs: ATLAS-3489
https://issues.apache.org/jira/browse/ATLAS-3489
Repository: atlas
Description (updated)
-------
Bug Details:
Input :
* Consider case where input zip contains hive_table which has 2 hive_columns with same qualifiedName but different entityStatus.
* One of the columns is ACTIVE while other is DELETED.
Expected Result :
Imported table must contain 2 columns with same qualifiedName but with different entityStatus.
Actual Result :
Imported table has 1 column with guid of ACTIVE column from input zip & entityStatus as DELETED.
Problem Area :
* Problem is in AtlasEntityStoreV2.preCreateOrUpdate().getResolvedEntityVertex(discoveryContext, entity), #951.
* This method fetches already resolved vertex for column with same qualifiedName since both entities have same uniqueAttributes, where as it should return null vertex & further new vertex should be created for entity with different entityStatus.
Solution :
* Added condition to retain entity's GUID when import is in progress.
Diffs
-----
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java 5b3c27dff
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java 95f6ec3bb
Diff: https://reviews.apache.org/r/71683/diff/1/
Testing
-------
Added new UT -> ImportServiceTest.testImportDuplicateColumnsWithDifferentStatus
Verified all existing UTs locally with the patch.
Tested manually.
Thanks,
Ashutosh Mestry
Re: Review Request 71683: ATLAS-3489: Import API: missing entity in
atlas after import
Posted by Nikhil Bonte <ni...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71683/#review218413
-----------------------------------------------------------
Ship it!
Ship It!
- Nikhil Bonte
On Oct. 26, 2019, 5:23 a.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71683/
> -----------------------------------------------------------
>
> (Updated Oct. 26, 2019, 5:23 a.m.)
>
>
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3489
> https://issues.apache.org/jira/browse/ATLAS-3489
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Bug Details:
>
> Input :
> * Consider case where input zip contains hive_table which has 2 hive_columns with same qualifiedName but different entityStatus.
> * One of the columns is ACTIVE while other is DELETED.
>
> Expected Result :
>
> Imported table must contain 2 columns with same qualifiedName but with different entityStatus.
>
> Actual Result :
>
> Imported table has 1 column with guid of ACTIVE column from input zip & entityStatus as DELETED.
>
>
> Problem Area :
>
> * Problem is in AtlasEntityStoreV2.preCreateOrUpdate().getResolvedEntityVertex(discoveryContext, entity), #951.
> * This method fetches already resolved vertex for column with same qualifiedName since both entities have same uniqueAttributes, where as it should return null vertex & further new vertex should be created for entity with different entityStatus.
>
> Solution :
>
> * Moved field 'entityStatus' from class AtlasRelatedObjectId to class AtlasObjectId.
> * Changed constructor of AtlasObjectId used in EntityGraphRetriever.toAtlasObjectId(AtlasEntity entity).
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java 5b3c27dff
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java 95f6ec3bb
>
>
> Diff: https://reviews.apache.org/r/71683/diff/1/
>
>
> Testing
> -------
>
> Added new UT -> ImportServiceTest.testImportDuplicateColumnsWithDifferentStatus
> Verified all existing UTs locally with the patch.
> Tested manually.
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 71683: ATLAS-3489: Import API: missing entity in
atlas after import
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71683/#review218414
-----------------------------------------------------------
- the comments in the review description are not in-sync with the fix, especially the following:
"Moved field 'entityStatus' from class AtlasRelatedObjectId to class AtlasObjectId."
- please update the tests to cover multiple entities in deleted status for the same uniq-attribute value i.e. 2 deleted, 1 active
- Madhan Neethiraj
On Oct. 26, 2019, 5:23 a.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71683/
> -----------------------------------------------------------
>
> (Updated Oct. 26, 2019, 5:23 a.m.)
>
>
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian.
>
>
> Bugs: ATLAS-3489
> https://issues.apache.org/jira/browse/ATLAS-3489
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Bug Details:
>
> Input :
> * Consider case where input zip contains hive_table which has 2 hive_columns with same qualifiedName but different entityStatus.
> * One of the columns is ACTIVE while other is DELETED.
>
> Expected Result :
>
> Imported table must contain 2 columns with same qualifiedName but with different entityStatus.
>
> Actual Result :
>
> Imported table has 1 column with guid of ACTIVE column from input zip & entityStatus as DELETED.
>
>
> Problem Area :
>
> * Problem is in AtlasEntityStoreV2.preCreateOrUpdate().getResolvedEntityVertex(discoveryContext, entity), #951.
> * This method fetches already resolved vertex for column with same qualifiedName since both entities have same uniqueAttributes, where as it should return null vertex & further new vertex should be created for entity with different entityStatus.
>
> Solution :
>
> * Moved field 'entityStatus' from class AtlasRelatedObjectId to class AtlasObjectId.
> * Changed constructor of AtlasObjectId used in EntityGraphRetriever.toAtlasObjectId(AtlasEntity entity).
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java 5b3c27dff
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java 95f6ec3bb
>
>
> Diff: https://reviews.apache.org/r/71683/diff/1/
>
>
> Testing
> -------
>
> Added new UT -> ImportServiceTest.testImportDuplicateColumnsWithDifferentStatus
> Verified all existing UTs locally with the patch.
> Tested manually.
>
>
> Thanks,
>
> Ashutosh Mestry
>
>