You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by on 2019/03/08 08:46:56 UTC
Review Request 70162: ATLAS-3067: updated hive types to remove use of
ownedRef/inverseRef attributes
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70162/
-----------------------------------------------------------
Review request for atlas, Ayub Pathan, Ashutosh Mestry, keval bhatt, Mehul Parikh, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sharmadha Sainath.
Bugs: ATLAS-3067
https://issues.apache.org/jira/browse/ATLAS-3067
Repository: atlas
Description
-------
Using recently added type-patch REMOVE_LEGACY_ATTRIBUTES, removed ownedRef/inverseRef attributes from hive entity-defs.
Diffs
-----
addons/models/1000-Hadoop/patches/008-remove-hive-legacy-attributes.json PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java d12284ef1
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipDefStoreV2.java 35d0577d0
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java 50559dda9
webapp/src/main/java/org/apache/atlas/notification/preprocessor/EntityPreprocessor.java 7eba27a49
webapp/src/main/java/org/apache/atlas/notification/preprocessor/HivePreprocessor.java d54c88def
webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java 0f95fbabc
Diff: https://reviews.apache.org/r/70162/diff/1/
Testing
-------
- verified that Hive entities created earlier (i.e. before applying type-patch 008-remove-hive-legacy-attributes.json) can be read/updated/deleted after the applying this patch
- verified create/update/delete of Hive entities succeed after applying this patch
- pre-commit tests run: https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/948/
Thanks,
madhan
Re: Review Request 70162: ATLAS-3067: updated hive types to remove
use of ownedRef/inverseRef attributes
Posted by .
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70162/
-----------------------------------------------------------
(Updated March 9, 2019, 1:29 a.m.)
Review request for atlas, Ayub Pathan, Ashutosh Mestry, keval bhatt, Mehul Parikh, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sharmadha Sainath.
Changes
-------
updated for the review comments.
Bugs: ATLAS-3067
https://issues.apache.org/jira/browse/ATLAS-3067
Repository: atlas
Description
-------
Using recently added type-patch REMOVE_LEGACY_ATTRIBUTES, removed ownedRef/inverseRef attributes from hive entity-defs.
Diffs (updated)
-----
addons/models/1000-Hadoop/patches/008-remove-hive-legacy-attributes.json PRE-CREATION
addons/models/2000-RDBMS/patches/003-remove-rdbms-legacy-attributes.json d087c660f
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java d12284ef1
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipDefStoreV2.java 35d0577d0
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java 50559dda9
webapp/src/main/java/org/apache/atlas/notification/preprocessor/EntityPreprocessor.java 7eba27a49
webapp/src/main/java/org/apache/atlas/notification/preprocessor/HivePreprocessor.java d54c88def
webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java 0f95fbabc
Diff: https://reviews.apache.org/r/70162/diff/2/
Changes: https://reviews.apache.org/r/70162/diff/1-2/
Testing
-------
- verified that Hive entities created earlier (i.e. before applying type-patch 008-remove-hive-legacy-attributes.json) can be read/updated/deleted after the applying this patch
- verified create/update/delete of Hive entities succeed after applying this patch
- pre-commit tests run: https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/948/
Thanks,
madhan
Re: Review Request 70162: ATLAS-3067: updated hive types to remove
use of ownedRef/inverseRef attributes
Posted by .
> On March 8, 2019, 8:59 p.m., Sarath Subramanian wrote:
> > addons/models/1000-Hadoop/patches/008-remove-hive-legacy-attributes.json
> > Lines 46 (patched)
> > <https://reviews.apache.org/r/70162/diff/1/?file=2129632#file2129632line46>
> >
> > why are we swapping endDefs here?
Once isLegacyAttribute flag was set to false for both ends, the edge created for the relationship is from end1->end2. This results in existing relationships to be ignored for hive_column_lineage->hive_process, as the order of ends don't match the edge direction. Hence this change is required.
- madhan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70162/#review213569
-----------------------------------------------------------
On March 9, 2019, 1:29 a.m., madhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70162/
> -----------------------------------------------------------
>
> (Updated March 9, 2019, 1:29 a.m.)
>
>
> Review request for atlas, Ayub Pathan, Ashutosh Mestry, keval bhatt, Mehul Parikh, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sharmadha Sainath.
>
>
> Bugs: ATLAS-3067
> https://issues.apache.org/jira/browse/ATLAS-3067
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Using recently added type-patch REMOVE_LEGACY_ATTRIBUTES, removed ownedRef/inverseRef attributes from hive entity-defs.
>
>
> Diffs
> -----
>
> addons/models/1000-Hadoop/patches/008-remove-hive-legacy-attributes.json PRE-CREATION
> addons/models/2000-RDBMS/patches/003-remove-rdbms-legacy-attributes.json d087c660f
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java d12284ef1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipDefStoreV2.java 35d0577d0
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java 50559dda9
> webapp/src/main/java/org/apache/atlas/notification/preprocessor/EntityPreprocessor.java 7eba27a49
> webapp/src/main/java/org/apache/atlas/notification/preprocessor/HivePreprocessor.java d54c88def
> webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java 0f95fbabc
>
>
> Diff: https://reviews.apache.org/r/70162/diff/2/
>
>
> Testing
> -------
>
> - verified that Hive entities created earlier (i.e. before applying type-patch 008-remove-hive-legacy-attributes.json) can be read/updated/deleted after the applying this patch
> - verified create/update/delete of Hive entities succeed after applying this patch
> - pre-commit tests run: https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/948/
>
>
> Thanks,
>
> madhan
>
>
Re: Review Request 70162: ATLAS-3067: updated hive types to remove
use of ownedRef/inverseRef attributes
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70162/#review213569
-----------------------------------------------------------
Fix it, then Ship it!
addons/models/1000-Hadoop/patches/008-remove-hive-legacy-attributes.json
Lines 6 (patched)
<https://reviews.apache.org/r/70162/#comment299544>
patch file in 007-hadoop_model_add_service_type.json updates all relationshipDefs versions to 1.1; consider updating applyToVersion and updateToVersion.
addons/models/1000-Hadoop/patches/008-remove-hive-legacy-attributes.json
Lines 40 (patched)
<https://reviews.apache.org/r/70162/#comment299545>
REMOVE_LEGACY_ATTRIBUTES => REMOVE_LEGACY_REFERENCE_ATTRIBUTES (considering we are removing only objectid attributes)
addons/models/1000-Hadoop/patches/008-remove-hive-legacy-attributes.json
Lines 46 (patched)
<https://reviews.apache.org/r/70162/#comment299546>
why are we swapping endDefs here?
- Sarath Subramanian
On March 8, 2019, 12:46 a.m., madhan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70162/
> -----------------------------------------------------------
>
> (Updated March 8, 2019, 12:46 a.m.)
>
>
> Review request for atlas, Ayub Pathan, Ashutosh Mestry, keval bhatt, Mehul Parikh, Nikhil Bonte, Nixon Rodrigues, Sarath Subramanian, and Sharmadha Sainath.
>
>
> Bugs: ATLAS-3067
> https://issues.apache.org/jira/browse/ATLAS-3067
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Using recently added type-patch REMOVE_LEGACY_ATTRIBUTES, removed ownedRef/inverseRef attributes from hive entity-defs.
>
>
> Diffs
> -----
>
> addons/models/1000-Hadoop/patches/008-remove-hive-legacy-attributes.json PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java d12284ef1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipDefStoreV2.java 35d0577d0
> webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java 50559dda9
> webapp/src/main/java/org/apache/atlas/notification/preprocessor/EntityPreprocessor.java 7eba27a49
> webapp/src/main/java/org/apache/atlas/notification/preprocessor/HivePreprocessor.java d54c88def
> webapp/src/main/java/org/apache/atlas/notification/preprocessor/PreprocessorContext.java 0f95fbabc
>
>
> Diff: https://reviews.apache.org/r/70162/diff/1/
>
>
> Testing
> -------
>
> - verified that Hive entities created earlier (i.e. before applying type-patch 008-remove-hive-legacy-attributes.json) can be read/updated/deleted after the applying this patch
> - verified create/update/delete of Hive entities succeed after applying this patch
> - pre-commit tests run: https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/948/
>
>
> Thanks,
>
> madhan
>
>