You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Sarath Subramanian <sa...@apache.org> on 2017/06/30 20:25:15 UTC
Review Request 60578: [ATLA-1907]: Create RelationshipDefs for all
base models and use this definitions when creating edges for entities
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/
-----------------------------------------------------------
Review request for atlas, David Radley and Madhan Neethiraj.
Bugs: ATLAS-1907
https://issues.apache.org/jira/browse/ATLAS-1907
Repository: atlas
Description
-------
Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
Diffs
-----
addons/models/0010-base_model.json 7f64d85e
addons/models/0030-hive_model.json b359ea82
addons/models/0050-falcon_model.json b7398ef4
addons/models/0060-hbase_model.json 42fe00c4
addons/models/0080-storm_model.json 095936e0
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 49ead6e3
intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java f85cf35c
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
Diff: https://reviews.apache.org/r/60578/diff/1/
Testing
-------
Tested using Postman REST client.
Validated all hive_table creation works fine and lineage is rendered right.
UTs/ITs in progress.
Thanks,
Sarath Subramanian
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by David Radley <da...@uk.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review179478
-----------------------------------------------------------
addons/models/0010-base_model.json
Lines 107 (patched)
<https://reviews.apache.org/r/60578/#comment254206>
why do we need to expose the legacy label value in the relationshipDef - can we not calculate it in the same way that the legacy code does. Is there any use case to have this label value different from the legacy values?
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 126 (patched)
<https://reviews.apache.org/r/60578/#comment254211>
What is the thinking behind issuing this warning? What is the user expected to change?
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 196 (patched)
<https://reviews.apache.org/r/60578/#comment254205>
remember to do the TODO
intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java
Lines 369 (patched)
<https://reviews.apache.org/r/60578/#comment254207>
remember the TODO
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 247 (patched)
<https://reviews.apache.org/r/60578/#comment254209>
check =>is
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 253 (patched)
<https://reviews.apache.org/r/60578/#comment254208>
I am not sure what this is doing. Why does each endDef having a legacy relation mean that there is a bidirectional relationship. Do you mean that the relaitonships is implmneted by 2 directional edges? This is not a bidirecitonal relationship (which we were thinking of as a single edge with direction BOTH).
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
Lines 409 (patched)
<https://reviews.apache.org/r/60578/#comment254210>
I see in the models we have columns defined in the old way and the new way - but this code seems to imply that without a relationshipDef we use the legacy way to create edges. I am unsure on the proposed design in this area.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
Lines 568 (patched)
<https://reviews.apache.org/r/60578/#comment254212>
reltionDef => relationshipDef
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
Lines 1157 (patched)
<https://reviews.apache.org/r/60578/#comment254213>
What are we calling the relationship attribute. Is this an attribute of the relationship or the name of the relationship or the name associated with the end. It would be good if we could agree on terms that we could use consistently through the code.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
Lines 1189 (patched)
<https://reviews.apache.org/r/60578/#comment254214>
atleast => at least
entry => Def
- David Radley
On June 30, 2017, 8:25 p.m., Sarath Subramanian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> -----------------------------------------------------------
>
> (Updated June 30, 2017, 8:25 p.m.)
>
>
> Review request for atlas, David Radley and Madhan Neethiraj.
>
>
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
>
>
> Diffs
> -----
>
> addons/models/0010-base_model.json 7f64d85e
> addons/models/0030-hive_model.json b359ea82
> addons/models/0050-falcon_model.json b7398ef4
> addons/models/0060-hbase_model.json 42fe00c4
> addons/models/0080-storm_model.json 095936e0
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 49ead6e3
> intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
> intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java f85cf35c
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
> intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/60578/diff/1/
>
>
> Testing
> -------
>
> Tested using Postman REST client.
> Validated all hive_table creation works fine and lineage is rendered right.
>
> UTs/ITs in progress.
>
>
> Thanks,
>
> Sarath Subramanian
>
>
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review179407
-----------------------------------------------------------
addons/models/0010-base_model.json
Lines 99 (patched)
<https://reviews.apache.org/r/60578/#comment254093>
Consider naming this as: dataset_process_inputs
Also, swap the endDefs: endDef1 -> dataset; endDef2 -> process. I think this would be intutive to read.
addons/models/0030-hive_model.json
Lines 527 (patched)
<https://reviews.apache.org/r/60578/#comment254094>
Consider the following:
- rename as 'hive_db_table' - from container to contained
- swap the endDefs: endDef1=hive_db; endDef2=hive_table
- propagateTags: TWO_TO_ONE ==> ONE_TO_TWO
addons/models/0030-hive_model.json
Lines 619 (patched)
<https://reviews.apache.org/r/60578/#comment254098>
I would suggest to keep the container end as endDef1.
addons/models/0050-falcon_model.json
Lines 162 (patched)
<https://reviews.apache.org/r/60578/#comment254099>
I would suggest to keep the container end as endDef1. Also, rename the relationship as "falcon_cluster_feed"
addons/models/0050-falcon_model.json
Lines 181 (patched)
<https://reviews.apache.org/r/60578/#comment254100>
I would suggest to keep the container end as endDef1. Also, rename the relationship as "falcon_cluster_process"
addons/models/0050-falcon_model.json
Lines 200 (patched)
<https://reviews.apache.org/r/60578/#comment254101>
I would suggest to keep the container end as endDef1. Also, rename the relationship as "falcon_cluster_feed_creation"
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 118 (patched)
<https://reviews.apache.org/r/60578/#comment254103>
add "@Override"
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java
Lines 147 (patched)
<https://reviews.apache.org/r/60578/#comment254105>
Consider making the following as unmodifiable:
- allSubTypes = Collections.unmodifiableSet(allSubTypes);
- typeAndAllSubTypes = Collections.unmodifiableSet(typeAndAllSubTypes);
- relationshipAttributes = Collections.unmodifiableMap(relationshipAttributes);
- relationshipAttributesType = Collections.unmodifiableMap(relationshipAttributesType);
Also, update accessors of these fields to simply return the member, instead of wrapping it with Collections.unmodifiable..()
intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java
Lines 368 (patched)
<https://reviews.apache.org/r/60578/#comment254106>
Please move these comments from here to appropriate file.
- Madhan Neethiraj
On June 30, 2017, 8:25 p.m., Sarath Subramanian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> -----------------------------------------------------------
>
> (Updated June 30, 2017, 8:25 p.m.)
>
>
> Review request for atlas, David Radley and Madhan Neethiraj.
>
>
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
>
>
> Diffs
> -----
>
> addons/models/0010-base_model.json 7f64d85e
> addons/models/0030-hive_model.json b359ea82
> addons/models/0050-falcon_model.json b7398ef4
> addons/models/0060-hbase_model.json 42fe00c4
> addons/models/0080-storm_model.json 095936e0
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 49ead6e3
> intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
> intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java f85cf35c
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
> intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/60578/diff/1/
>
>
> Testing
> -------
>
> Tested using Postman REST client.
> Validated all hive_table creation works fine and lineage is rendered right.
>
> UTs/ITs in progress.
>
>
> Thanks,
>
> Sarath Subramanian
>
>
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review179423
-----------------------------------------------------------
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 353 (patched)
<https://reviews.apache.org/r/60578/#comment254120>
"if" ==> "else if"
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 377 (patched)
<https://reviews.apache.org/r/60578/#comment254121>
for types that don't have superTypes, this method would return null - and cause NPE in #347/#348/#353/#354 Instead of computing this List (why not Set?) every time, consider adding a method to appropriate AtlasEntityType.
- Madhan Neethiraj
On June 30, 2017, 8:25 p.m., Sarath Subramanian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> -----------------------------------------------------------
>
> (Updated June 30, 2017, 8:25 p.m.)
>
>
> Review request for atlas, David Radley and Madhan Neethiraj.
>
>
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
>
>
> Diffs
> -----
>
> addons/models/0010-base_model.json 7f64d85e
> addons/models/0030-hive_model.json b359ea82
> addons/models/0050-falcon_model.json b7398ef4
> addons/models/0060-hbase_model.json 42fe00c4
> addons/models/0080-storm_model.json 095936e0
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 49ead6e3
> intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
> intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java f85cf35c
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
> intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/60578/diff/1/
>
>
> Testing
> -------
>
> Tested using Postman REST client.
> Validated all hive_table creation works fine and lineage is rendered right.
>
> UTs/ITs in progress.
>
>
> Thanks,
>
> Sarath Subramanian
>
>
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review180183
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On July 11, 2017, 7:40 a.m., Sarath Subramanian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> -----------------------------------------------------------
>
> (Updated July 11, 2017, 7:40 a.m.)
>
>
> Review request for atlas, David Radley and Madhan Neethiraj.
>
>
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
>
>
> Diffs
> -----
>
> addons/models/0010-base_model.json 7f64d85e
> addons/models/0030-hive_model.json b359ea82
> addons/models/0050-falcon_model.json b7398ef4
> addons/models/0060-hbase_model.json 42fe00c4
> addons/models/0080-storm_model.json 095936e0
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java e8afed1d
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 0e277b19
> intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
> intg/src/main/java/org/apache/atlas/model/instance/AtlasStruct.java 7c8ae2d7
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
> intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 49a94931
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
> intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8
> intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
> repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 329dd7a4
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java a5b5730f
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/60578/diff/6/
>
>
> Testing
> -------
>
> mvn clean package succeeded. Validated hive_table creation and lineage rendering.
>
> UTs/ITs in progress.
>
>
> Thanks,
>
> Sarath Subramanian
>
>
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/
-----------------------------------------------------------
(Updated July 11, 2017, 12:40 a.m.)
Review request for atlas, David Radley and Madhan Neethiraj.
Bugs: ATLAS-1907
https://issues.apache.org/jira/browse/ATLAS-1907
Repository: atlas
Description
-------
Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
Diffs (updated)
-----
addons/models/0010-base_model.json 7f64d85e
addons/models/0030-hive_model.json b359ea82
addons/models/0050-falcon_model.json b7398ef4
addons/models/0060-hbase_model.json 42fe00c4
addons/models/0080-storm_model.json 095936e0
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java e8afed1d
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 0e277b19
intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
intg/src/main/java/org/apache/atlas/model/instance/AtlasStruct.java 7c8ae2d7
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 49a94931
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8
intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 329dd7a4
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java a5b5730f
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
Diff: https://reviews.apache.org/r/60578/diff/6/
Changes: https://reviews.apache.org/r/60578/diff/5-6/
Testing
-------
mvn clean package succeeded. Validated hive_table creation and lineage rendering.
UTs/ITs in progress.
Thanks,
Sarath Subramanian
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/
-----------------------------------------------------------
(Updated July 10, 2017, 5:09 p.m.)
Review request for atlas, David Radley and Madhan Neethiraj.
Changes
-------
fixed UT failures.
Bugs: ATLAS-1907
https://issues.apache.org/jira/browse/ATLAS-1907
Repository: atlas
Description
-------
Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
Diffs (updated)
-----
addons/models/0010-base_model.json 7f64d85e
addons/models/0030-hive_model.json b359ea82
addons/models/0050-falcon_model.json b7398ef4
addons/models/0060-hbase_model.json 42fe00c4
addons/models/0080-storm_model.json 095936e0
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java e8afed1d
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 0e277b19
intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
intg/src/main/java/org/apache/atlas/model/instance/AtlasStruct.java 7c8ae2d7
intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java fc820d49
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 49a94931
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8
intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 329dd7a4
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java a5b5730f
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java 2ac0fc66
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
Diff: https://reviews.apache.org/r/60578/diff/4/
Changes: https://reviews.apache.org/r/60578/diff/3-4/
Testing (updated)
-------
mvn clean package succeeded. Validated hive_table creation and lineage is rendering.
UTs/ITs in progress.
Thanks,
Sarath Subramanian
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/
-----------------------------------------------------------
(Updated July 8, 2017, 10:08 p.m.)
Review request for atlas, David Radley and Madhan Neethiraj.
Changes
-------
addressed review comments:
. made relationshipAttributes and relationshipAttributeTypes as unmodifiable map
Bugs: ATLAS-1907
https://issues.apache.org/jira/browse/ATLAS-1907
Repository: atlas
Description
-------
Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
Diffs (updated)
-----
addons/models/0010-base_model.json 7f64d85e
addons/models/0030-hive_model.json b359ea82
addons/models/0050-falcon_model.json b7398ef4
addons/models/0060-hbase_model.json 42fe00c4
addons/models/0080-storm_model.json 095936e0
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 73dd33aa
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 0e277b19
intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
intg/src/main/java/org/apache/atlas/model/instance/AtlasStruct.java 7c8ae2d7
intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java fc820d49
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 49a94931
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8
intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 329dd7a4
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java a5b5730f
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
Diff: https://reviews.apache.org/r/60578/diff/3/
Changes: https://reviews.apache.org/r/60578/diff/2-3/
Testing
-------
Tested using Postman REST client.
Validated all hive_table creation works fine and lineage is rendered right.
UTs/ITs in progress.
Thanks,
Sarath Subramanian
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by Sarath Subramanian <sa...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/
-----------------------------------------------------------
(Updated July 6, 2017, 5:12 p.m.)
Review request for atlas, David Radley and Madhan Neethiraj.
Changes
-------
addressed review comments
Bugs: ATLAS-1907
https://issues.apache.org/jira/browse/ATLAS-1907
Repository: atlas
Description
-------
Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
Diffs (updated)
-----
addons/models/0010-base_model.json 7f64d85e
addons/models/0030-hive_model.json b359ea82
addons/models/0050-falcon_model.json b7398ef4
addons/models/0060-hbase_model.json 42fe00c4
addons/models/0080-storm_model.json 095936e0
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 73dd33aa
intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 49a94931
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
intg/src/main/java/org/apache/atlas/type/AtlasType.java 86072fe8
intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
Diff: https://reviews.apache.org/r/60578/diff/2/
Changes: https://reviews.apache.org/r/60578/diff/1-2/
Testing
-------
Tested using Postman REST client.
Validated all hive_table creation works fine and lineage is rendered right.
UTs/ITs in progress.
Thanks,
Sarath Subramanian
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by Mandy Chessell <ma...@uk.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review179484
-----------------------------------------------------------
Please rename the following JSON files to permit the common model (https://issues.apache.org/jira/browse/ATLAS-1768) to be inserted between 0010 Base Model and the Hadoop models.
* 1020 FS Model
* 1030 Hive Model
* 1040 Sqoop Model
* 1050 Falcon Model
* 1060 HBase Model
* 1070 Kafka Model
* 1080 Storm Model
- Mandy Chessell
On June 30, 2017, 8:25 p.m., Sarath Subramanian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> -----------------------------------------------------------
>
> (Updated June 30, 2017, 8:25 p.m.)
>
>
> Review request for atlas, David Radley and Madhan Neethiraj.
>
>
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
>
>
> Diffs
> -----
>
> addons/models/0010-base_model.json 7f64d85e
> addons/models/0030-hive_model.json b359ea82
> addons/models/0050-falcon_model.json b7398ef4
> addons/models/0060-hbase_model.json 42fe00c4
> addons/models/0080-storm_model.json 095936e0
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 49ead6e3
> intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
> intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java f85cf35c
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
> intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/60578/diff/1/
>
>
> Testing
> -------
>
> Tested using Postman REST client.
> Validated all hive_table creation works fine and lineage is rendered right.
>
> UTs/ITs in progress.
>
>
> Thanks,
>
> Sarath Subramanian
>
>
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by David Radley <da...@uk.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review179471
-----------------------------------------------------------
addons/models/0030-hive_model.json
Lines 551 (patched)
<https://reviews.apache.org/r/60578/#comment254185>
I am unsure how this is working. We have columns defined in the old style with constraints and we have them defined in the new style. I was not expecting 2 definitions for columns. Isn't this going to show in the graph as the old style EntityDef vertices and edges and the new style relationshipDef and edges.
- David Radley
On June 30, 2017, 8:25 p.m., Sarath Subramanian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> -----------------------------------------------------------
>
> (Updated June 30, 2017, 8:25 p.m.)
>
>
> Review request for atlas, David Radley and Madhan Neethiraj.
>
>
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
>
>
> Diffs
> -----
>
> addons/models/0010-base_model.json 7f64d85e
> addons/models/0030-hive_model.json b359ea82
> addons/models/0050-falcon_model.json b7398ef4
> addons/models/0060-hbase_model.json 42fe00c4
> addons/models/0080-storm_model.json 095936e0
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 49ead6e3
> intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
> intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java f85cf35c
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
> intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/60578/diff/1/
>
>
> Testing
> -------
>
> Tested using Postman REST client.
> Validated all hive_table creation works fine and lineage is rendered right.
>
> UTs/ITs in progress.
>
>
> Thanks,
>
> Sarath Subramanian
>
>
Re: Review Request 60578: [ATLA-1907]: Create RelationshipDefs for
all base
models and use this definitions when creating edges for entities
Posted by Mandy Chessell <ma...@uk.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60578/#review179494
-----------------------------------------------------------
The Atlas WIKI has UML diagrams of each of the current models. They are located on child pages under https://cwiki.apache.org/confluence/display/ATLAS/Atlas+Model
I am happy to update the diagrams is you create me a subtask for this Jira
- Mandy Chessell
On June 30, 2017, 8:25 p.m., Sarath Subramanian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60578/
> -----------------------------------------------------------
>
> (Updated June 30, 2017, 8:25 p.m.)
>
>
> Review request for atlas, David Radley and Madhan Neethiraj.
>
>
> Bugs: ATLAS-1907
> https://issues.apache.org/jira/browse/ATLAS-1907
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Create relationshipDefs for all out of the box models. Use relationshipDef information when creating edges for composite attributes in entities.
>
>
> Diffs
> -----
>
> addons/models/0010-base_model.json 7f64d85e
> addons/models/0030-hive_model.json b359ea82
> addons/models/0050-falcon_model.json b7398ef4
> addons/models/0060-hbase_model.json 42fe00c4
> addons/models/0080-storm_model.json 095936e0
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 49ead6e3
> intg/src/main/java/org/apache/atlas/model/instance/AtlasRelationship.java 8d2e7ec5
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 70e3067a
> intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java f85cf35c
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java c2e0be50
> intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 29fae1c8
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java 4530f51b
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 7b3f1e6e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java 86212338
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java ebf6a201
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/60578/diff/1/
>
>
> Testing
> -------
>
> Tested using Postman REST client.
> Validated all hive_table creation works fine and lineage is rendered right.
>
> UTs/ITs in progress.
>
>
> Thanks,
>
> Sarath Subramanian
>
>