You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ashutosh Mestry via Review Board <no...@reviews.apache.org> on 2020/11/04 02:35:06 UTC
Re: Review Request 72991: Export/Import: Add Support for
BusinessMetadata Attributes
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72991/
-----------------------------------------------------------
(Updated Nov. 4, 2020, 2:35 a.m.)
Review request for atlas, Nikhil Bonte and Sarath Subramanian.
Changes
-------
Updates include: Addressed review comments.
Bugs: ATLAS-4006 and ATLAS-4007
https://issues.apache.org/jira/browse/ATLAS-4006
https://issues.apache.org/jira/browse/ATLAS-4007
Repository: atlas
Description
-------
**Background**
Existing implementation does not support export and import of _Business Metadata_ (BM) attributes.
**Implementation**
- Export: Updated existing flow to include detecting BM attributes and adding the types to the exported payload.
- Import: Updated _TypeAttributeDifference_ to include BM attribute detection and update.
Diffs (updated)
-----
repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 0491a853a
repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java 664af9166
repository/src/main/java/org/apache/atlas/repository/impexp/ImportTypeDefProcessor.java b65807742
repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java 0d78dbe57
repository/src/test/java/org/apache/atlas/repository/impexp/TypeAttributeDifferenceTest.java f9b7cd11f
repository/src/test/resources/json/typesDef-bm.json PRE-CREATION
Diff: https://reviews.apache.org/r/72991/diff/2/
Changes: https://reviews.apache.org/r/72991/diff/1-2/
Testing
-------
**Unit tests**
New tests added.
**Functional testing**
End-to-end tests. (Export-import-export).
**Pre-commit Build**
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/154/
File Attachments (updated)
----------------
Stocks with BM
https://reviews.apache.org/media/uploaded/files/2020/10/29/8ea9511f-3b0e-4b27-96cb-e5f2f50ba75d__stocks-bm.zip
Thanks,
Ashutosh Mestry
Re: Review Request 72991: Export/Import: Add Support for
BusinessMetadata Attributes
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72991/#review222170
-----------------------------------------------------------
repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java
Lines 185 (patched)
<https://reviews.apache.org/r/72991/#comment311235>
Consider replacing #184 - #186 with:
for (String bmTypeName : entityType.getBusinessAttributes().keySet()) {
AtlasBusinessMetadataType bmType = typeRegistry.getBusinessMetadataTypeByName(bmTypeName);
addBusinessMetadataType(bmType, context);
}
This will ensure that attribute-types of the business-metadata type will be processed as well (#179), like enum type attribute.
- Madhan Neethiraj
On Nov. 4, 2020, 2:35 a.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72991/
> -----------------------------------------------------------
>
> (Updated Nov. 4, 2020, 2:35 a.m.)
>
>
> Review request for atlas, Nikhil Bonte and Sarath Subramanian.
>
>
> Bugs: ATLAS-4006 and ATLAS-4007
> https://issues.apache.org/jira/browse/ATLAS-4006
> https://issues.apache.org/jira/browse/ATLAS-4007
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Background**
> Existing implementation does not support export and import of _Business Metadata_ (BM) attributes.
>
> **Implementation**
> - Export: Updated existing flow to include detecting BM attributes and adding the types to the exported payload.
> - Import: Updated _TypeAttributeDifference_ to include BM attribute detection and update.
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 0491a853a
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java 664af9166
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTypeDefProcessor.java b65807742
> repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java 0d78dbe57
> repository/src/test/java/org/apache/atlas/repository/impexp/TypeAttributeDifferenceTest.java f9b7cd11f
> repository/src/test/resources/json/typesDef-bm.json PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/72991/diff/2/
>
>
> Testing
> -------
>
> **Unit tests**
> New tests added.
>
> **Functional testing**
> End-to-end tests. (Export-import-export).
>
> **Pre-commit Build**
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/154/
>
>
> File Attachments
> ----------------
>
> Stocks with BM
> https://reviews.apache.org/media/uploaded/files/2020/10/29/8ea9511f-3b0e-4b27-96cb-e5f2f50ba75d__stocks-bm.zip
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 72991: Export/Import: Add Support for
BusinessMetadata Attributes
Posted by Nikhil Bonte <ni...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72991/#review222187
-----------------------------------------------------------
Ship it!
Ship It!
- Nikhil Bonte
On Nov. 10, 2020, 12:21 a.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72991/
> -----------------------------------------------------------
>
> (Updated Nov. 10, 2020, 12:21 a.m.)
>
>
> Review request for atlas, Nikhil Bonte and Sarath Subramanian.
>
>
> Bugs: ATLAS-4006 and ATLAS-4007
> https://issues.apache.org/jira/browse/ATLAS-4006
> https://issues.apache.org/jira/browse/ATLAS-4007
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Background**
> Existing implementation does not support export and import of _Business Metadata_ (BM) attributes.
>
> **Implementation**
> - Export: Updated existing flow to include detecting BM attributes and adding the types to the exported payload.
> - Import: Updated _TypeAttributeDifference_ to include BM attribute detection and update.
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/impexp/AtlasServerService.java 542106f36
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 0491a853a
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java 664af9166
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTypeDefProcessor.java b65807742
> repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java 0d78dbe57
> repository/src/test/java/org/apache/atlas/repository/impexp/TypeAttributeDifferenceTest.java f9b7cd11f
> repository/src/test/resources/json/typesDef-bm.json PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/72991/diff/3/
>
>
> Testing
> -------
>
> **Unit tests**
> New tests added.
>
> **Functional testing**
> End-to-end tests. (Export-import-export).
>
> **Pre-commit Build**
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/154/
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/179/
>
>
> File Attachments
> ----------------
>
> Stocks with BM
> https://reviews.apache.org/media/uploaded/files/2020/10/29/8ea9511f-3b0e-4b27-96cb-e5f2f50ba75d__stocks-bm.zip
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 72991: Export/Import: Add Support for
BusinessMetadata Attributes
Posted by Ashutosh Mestry via Review Board <no...@reviews.apache.org>.
> On Nov. 10, 2020, 6:02 a.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/repository/impexp/AtlasServerService.java
> > Line 108 (original), 108 (patched)
> > <https://reviews.apache.org/r/72991/diff/3/?file=2241846#file2241846line108>
> >
> > why ignore relationships flag is turned ON?
This is a use case where all the imported entities should have the _replicatedFrom_ attribute set. This https://issues.apache.org/jira/browse/ATLAS-3109 inadvertently removed that behavior causing only the top-level entities to have that attribite.
- Ashutosh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72991/#review222183
-----------------------------------------------------------
On Nov. 10, 2020, 12:21 a.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72991/
> -----------------------------------------------------------
>
> (Updated Nov. 10, 2020, 12:21 a.m.)
>
>
> Review request for atlas, Nikhil Bonte and Sarath Subramanian.
>
>
> Bugs: ATLAS-4006 and ATLAS-4007
> https://issues.apache.org/jira/browse/ATLAS-4006
> https://issues.apache.org/jira/browse/ATLAS-4007
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Background**
> Existing implementation does not support export and import of _Business Metadata_ (BM) attributes.
>
> **Implementation**
> - Export: Updated existing flow to include detecting BM attributes and adding the types to the exported payload.
> - Import: Updated _TypeAttributeDifference_ to include BM attribute detection and update.
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/impexp/AtlasServerService.java 542106f36
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 0491a853a
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java 664af9166
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTypeDefProcessor.java b65807742
> repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java 0d78dbe57
> repository/src/test/java/org/apache/atlas/repository/impexp/TypeAttributeDifferenceTest.java f9b7cd11f
> repository/src/test/resources/json/typesDef-bm.json PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/72991/diff/3/
>
>
> Testing
> -------
>
> **Unit tests**
> New tests added.
>
> **Functional testing**
> End-to-end tests. (Export-import-export).
>
> **Pre-commit Build**
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/154/
>
>
> File Attachments
> ----------------
>
> Stocks with BM
> https://reviews.apache.org/media/uploaded/files/2020/10/29/8ea9511f-3b0e-4b27-96cb-e5f2f50ba75d__stocks-bm.zip
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 72991: Export/Import: Add Support for
BusinessMetadata Attributes
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72991/#review222183
-----------------------------------------------------------
Fix it, then Ship it!
repository/src/main/java/org/apache/atlas/repository/impexp/AtlasServerService.java
Line 108 (original), 108 (patched)
<https://reviews.apache.org/r/72991/#comment311241>
why ignore relationships flag is turned ON?
- Sarath Subramanian
On Nov. 9, 2020, 4:21 p.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72991/
> -----------------------------------------------------------
>
> (Updated Nov. 9, 2020, 4:21 p.m.)
>
>
> Review request for atlas, Nikhil Bonte and Sarath Subramanian.
>
>
> Bugs: ATLAS-4006 and ATLAS-4007
> https://issues.apache.org/jira/browse/ATLAS-4006
> https://issues.apache.org/jira/browse/ATLAS-4007
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Background**
> Existing implementation does not support export and import of _Business Metadata_ (BM) attributes.
>
> **Implementation**
> - Export: Updated existing flow to include detecting BM attributes and adding the types to the exported payload.
> - Import: Updated _TypeAttributeDifference_ to include BM attribute detection and update.
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/impexp/AtlasServerService.java 542106f36
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 0491a853a
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java 664af9166
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTypeDefProcessor.java b65807742
> repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java 0d78dbe57
> repository/src/test/java/org/apache/atlas/repository/impexp/TypeAttributeDifferenceTest.java f9b7cd11f
> repository/src/test/resources/json/typesDef-bm.json PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/72991/diff/3/
>
>
> Testing
> -------
>
> **Unit tests**
> New tests added.
>
> **Functional testing**
> End-to-end tests. (Export-import-export).
>
> **Pre-commit Build**
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/154/
>
>
> File Attachments
> ----------------
>
> Stocks with BM
> https://reviews.apache.org/media/uploaded/files/2020/10/29/8ea9511f-3b0e-4b27-96cb-e5f2f50ba75d__stocks-bm.zip
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 72991: Export/Import: Add Support for
BusinessMetadata Attributes
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72991/#review222186
-----------------------------------------------------------
Ship it!
Ship It!
- Sarath Subramanian
On Nov. 9, 2020, 4:21 p.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72991/
> -----------------------------------------------------------
>
> (Updated Nov. 9, 2020, 4:21 p.m.)
>
>
> Review request for atlas, Nikhil Bonte and Sarath Subramanian.
>
>
> Bugs: ATLAS-4006 and ATLAS-4007
> https://issues.apache.org/jira/browse/ATLAS-4006
> https://issues.apache.org/jira/browse/ATLAS-4007
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Background**
> Existing implementation does not support export and import of _Business Metadata_ (BM) attributes.
>
> **Implementation**
> - Export: Updated existing flow to include detecting BM attributes and adding the types to the exported payload.
> - Import: Updated _TypeAttributeDifference_ to include BM attribute detection and update.
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/impexp/AtlasServerService.java 542106f36
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 0491a853a
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java 664af9166
> repository/src/main/java/org/apache/atlas/repository/impexp/ImportTypeDefProcessor.java b65807742
> repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java 0d78dbe57
> repository/src/test/java/org/apache/atlas/repository/impexp/TypeAttributeDifferenceTest.java f9b7cd11f
> repository/src/test/resources/json/typesDef-bm.json PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/72991/diff/3/
>
>
> Testing
> -------
>
> **Unit tests**
> New tests added.
>
> **Functional testing**
> End-to-end tests. (Export-import-export).
>
> **Pre-commit Build**
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/154/
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/179/
>
>
> File Attachments
> ----------------
>
> Stocks with BM
> https://reviews.apache.org/media/uploaded/files/2020/10/29/8ea9511f-3b0e-4b27-96cb-e5f2f50ba75d__stocks-bm.zip
>
>
> Thanks,
>
> Ashutosh Mestry
>
>
Re: Review Request 72991: Export/Import: Add Support for
BusinessMetadata Attributes
Posted by Ashutosh Mestry via Review Board <no...@reviews.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72991/
-----------------------------------------------------------
(Updated Nov. 10, 2020, 12:21 a.m.)
Review request for atlas, Nikhil Bonte and Sarath Subramanian.
Changes
-------
Updates include:
- Addressed review comments.
- Fixed bug introduced by https://issues.apache.org/jira/browse/ATLAS-3109
Bugs: ATLAS-4006 and ATLAS-4007
https://issues.apache.org/jira/browse/ATLAS-4006
https://issues.apache.org/jira/browse/ATLAS-4007
Repository: atlas
Description
-------
**Background**
Existing implementation does not support export and import of _Business Metadata_ (BM) attributes.
**Implementation**
- Export: Updated existing flow to include detecting BM attributes and adding the types to the exported payload.
- Import: Updated _TypeAttributeDifference_ to include BM attribute detection and update.
Diffs (updated)
-----
repository/src/main/java/org/apache/atlas/repository/impexp/AtlasServerService.java 542106f36
repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 0491a853a
repository/src/main/java/org/apache/atlas/repository/impexp/ExportTypeProcessor.java 664af9166
repository/src/main/java/org/apache/atlas/repository/impexp/ImportTypeDefProcessor.java b65807742
repository/src/main/java/org/apache/atlas/repository/impexp/TypeAttributeDifference.java 0d78dbe57
repository/src/test/java/org/apache/atlas/repository/impexp/TypeAttributeDifferenceTest.java f9b7cd11f
repository/src/test/resources/json/typesDef-bm.json PRE-CREATION
Diff: https://reviews.apache.org/r/72991/diff/3/
Changes: https://reviews.apache.org/r/72991/diff/2-3/
Testing
-------
**Unit tests**
New tests added.
**Functional testing**
End-to-end tests. (Export-import-export).
**Pre-commit Build**
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/154/
File Attachments
----------------
Stocks with BM
https://reviews.apache.org/media/uploaded/files/2020/10/29/8ea9511f-3b0e-4b27-96cb-e5f2f50ba75d__stocks-bm.zip
Thanks,
Ashutosh Mestry