You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Suma Shivaprasad <su...@gmail.com> on 2017/01/14 03:57:49 UTC
Review Request 55531: ATLAS-1467 Create/Full Update implementation
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/
-----------------------------------------------------------
Review request for atlas.
Bugs: ATLAS-1467
https://issues.apache.org/jira/browse/ATLAS-1467
Repository: atlas
Description
-------
Implemented
1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
2. Create with new APIs and POJOs ( no change in graph model )
3. Full Update with new APIs and POJOs ( no change in graph model )
Diffs
-----
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 6770c41
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
intg/src/main/java/org/apache/atlas/model/typedef/AtlasEntityDef.java 85e9d77
intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java be9fed2
repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java bcdf344
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 7444bf3
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
Diff: https://reviews.apache.org/r/55531/diff/
Testing
-------
UTs for create
TODO:
Update testing pending
Error handling for unresolved references
UTS for discovery
Thanks,
Suma Shivaprasad
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/#review162247
-----------------------------------------------------------
intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java (line 99)
<https://reviews.apache.org/r/55531/#comment233562>
This method is already defined in the base class - AtlasStructType. Consider removing it from here.
intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java (line 102)
<https://reviews.apache.org/r/55531/#comment233563>
This override simply calls the base class impl. Consider removing it from here.
intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java (line 129)
<https://reviews.apache.org/r/55531/#comment233567>
Since the base class has 'allAttributes', can getAttributeDef() be moved to the base class?
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java (line 73)
<https://reviews.apache.org/r/55531/#comment233564>
is "allAd" needed here?
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java (line 101)
<https://reviews.apache.org/r/55531/#comment233565>
This method is already defined in the base class - AtlasStructType. Consider removing it from here.
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java (line 104)
<https://reviews.apache.org/r/55531/#comment233566>
This override simply calls the base class impl. Consider removing from here.
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java (line 126)
<https://reviews.apache.org/r/55531/#comment233568>
Since the base class has 'allAttributes', can getAttributeDef() be moved to the base class?
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java (line 270)
<https://reviews.apache.org/r/55531/#comment233569>
'allAttributeDefs' argument may not be necessary. Please review.
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java (line 80)
<https://reviews.apache.org/r/55531/#comment233570>
Shouldn't this handle 'attributeName' from superTypes as well - by looking into 'allAttributes', instead of 'structDef'.
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java (line 516)
<https://reviews.apache.org/r/55531/#comment233571>
Consider taking only the following 2 arguments, as the rest can be derived from these:
AtlasAttribute(AtlasStructType structType, String attributeName) {
this.structType = structType;
this.structDef = structType.getStructDef();
this.attributeDef = structType.getAttributeDef(attributeName);
this.attributeType = structType.getAttributeType(attributeName);
this.qualifiedName = getQualifiedAttributeName(structDef, attributeName);
}
- Madhan Neethiraj
On Jan. 19, 2017, 6:22 a.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55531/
> -----------------------------------------------------------
>
> (Updated Jan. 19, 2017, 6:22 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1467
> https://issues.apache.org/jira/browse/ATLAS-1467
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Implemented
>
> 1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
> 2. Create with new APIs and POJOs ( no change in graph model )
> 3. Full Update with new APIs and POJOs ( no change in graph model )
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 6770c41
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasEntityDef.java 85e9d77
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
> intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
> intg/src/test/java/org/apache/atlas/type/TestAtlasTypeRegistry.java 9429c07
> repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 2be9a2d
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
> repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 5259249
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
> repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java c7c3286
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
> server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java c4be236
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java 3565ab3
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
> webapp/src/main/java/org/apache/atlas/web/util/LineageUtils.java 54ca236
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 265b650
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java 2a75773
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java dcb1264
> webapp/src/test/java/org/apache/atlas/web/resources/EntityDiscoveryJerseyResourceIT.java 2ade5b0
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d7702e2
>
> Diff: https://reviews.apache.org/r/55531/diff/
>
>
> Testing
> -------
>
> UTs for create
>
> TODO:
> Update testing pending
> Error handling for unresolved references
> UTS for discovery
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/#review162378
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Jan. 19, 2017, 10:13 p.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55531/
> -----------------------------------------------------------
>
> (Updated Jan. 19, 2017, 10:13 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1467
> https://issues.apache.org/jira/browse/ATLAS-1467
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Implemented
>
> 1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
> 2. Create with new APIs and POJOs ( no change in graph model )
> 3. Full Update with new APIs and POJOs ( no change in graph model )
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java a6438ed
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
> intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
> intg/src/test/java/org/apache/atlas/type/TestAtlasTypeRegistry.java 9429c07
> repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 2be9a2d
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
> repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 5259249
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
> repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java c7c3286
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
> server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java c4be236
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java 3565ab3
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
> webapp/src/main/java/org/apache/atlas/web/util/LineageUtils.java 54ca236
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 265b650
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java 2a75773
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 44d8a11
> webapp/src/test/java/org/apache/atlas/web/resources/EntityDiscoveryJerseyResourceIT.java 2ade5b0
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d7702e2
>
> Diff: https://reviews.apache.org/r/55531/diff/
>
>
> Testing
> -------
>
> UTs for create
>
> TODO:
> Update testing pending
> Error handling for unresolved references
> UTS for discovery
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Suma Shivaprasad <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/
-----------------------------------------------------------
(Updated Jan. 19, 2017, 10:13 p.m.)
Review request for atlas.
Changes
-------
Fixed review comment in AtlasClassificationType
Bugs: ATLAS-1467
https://issues.apache.org/jira/browse/ATLAS-1467
Repository: atlas
Description
-------
Implemented
1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
2. Create with new APIs and POJOs ( no change in graph model )
3. Full Update with new APIs and POJOs ( no change in graph model )
Diffs (updated)
-----
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java a6438ed
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
intg/src/test/java/org/apache/atlas/type/TestAtlasTypeRegistry.java 9429c07
repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 2be9a2d
repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 5259249
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java c7c3286
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java c4be236
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java 3565ab3
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
webapp/src/main/java/org/apache/atlas/web/util/LineageUtils.java 54ca236
webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 265b650
webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java 2a75773
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 44d8a11
webapp/src/test/java/org/apache/atlas/web/resources/EntityDiscoveryJerseyResourceIT.java 2ade5b0
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d7702e2
Diff: https://reviews.apache.org/r/55531/diff/
Testing
-------
UTs for create
TODO:
Update testing pending
Error handling for unresolved references
UTS for discovery
Thanks,
Suma Shivaprasad
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Suma Shivaprasad <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/
-----------------------------------------------------------
(Updated Jan. 19, 2017, 10:08 p.m.)
Review request for atlas.
Changes
-------
Fixed review comments
Bugs: ATLAS-1467
https://issues.apache.org/jira/browse/ATLAS-1467
Repository: atlas
Description
-------
Implemented
1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
2. Create with new APIs and POJOs ( no change in graph model )
3. Full Update with new APIs and POJOs ( no change in graph model )
Diffs (updated)
-----
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java a6438ed
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
intg/src/test/java/org/apache/atlas/type/TestAtlasTypeRegistry.java 9429c07
repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 2be9a2d
repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 5259249
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java c7c3286
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java c4be236
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java 3565ab3
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
webapp/src/main/java/org/apache/atlas/web/util/LineageUtils.java 54ca236
webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 265b650
webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java 2a75773
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 44d8a11
webapp/src/test/java/org/apache/atlas/web/resources/EntityDiscoveryJerseyResourceIT.java 2ade5b0
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d7702e2
Diff: https://reviews.apache.org/r/55531/diff/
Testing
-------
UTs for create
TODO:
Update testing pending
Error handling for unresolved references
UTS for discovery
Thanks,
Suma Shivaprasad
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/#review162252
-----------------------------------------------------------
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java (line 264)
<https://reviews.apache.org/r/55531/#comment233582>
I think 'string' should not be a valid value for a Struct/Entity/Classification. If there is no usecase to allow this value, please remove this.
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java (line 266)
<https://reviews.apache.org/r/55531/#comment233583>
AtlasObjectId should be allowed only when attribute-type is a entity. It wouldn't be a valid value for struct or classficaition. Please review.
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java (line 111)
<https://reviews.apache.org/r/55531/#comment233589>
unresolvedEntityReferences is type List<AtlasEntity>. would calling remove(AtlasObjectId id) result in correct removal? Please review.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java (line 56)
<https://reviews.apache.org/r/55531/#comment233602>
Consider marking graph, mapVertexMapper and arrVertexMapper as final.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java (line 157)
<https://reviews.apache.org/r/55531/#comment233603>
Should vertex be cleaned of properties that are not present in value?
updateTime, updatedBy system attributes?
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java (line 174)
<https://reviews.apache.org/r/55531/#comment233604>
Shouldn't createdBy and updatedBy attributes be set here?
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java (line 46)
<https://reviews.apache.org/r/55531/#comment233596>
Consider marking 'typeRegistry' and 'graphHelper' as final.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java (line 55)
<https://reviews.apache.org/r/55531/#comment233598>
Consider moving this to line #49, along with other attributes.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java (line 93)
<https://reviews.apache.org/r/55531/#comment233599>
after the validation in line #83, there shouldn't be any unresolvedEntityReference here, right?
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java (line 107)
<https://reviews.apache.org/r/55531/#comment233600>
String qualifiedAttrName = attr.getQualifiedName();
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java (line 114)
<https://reviews.apache.org/r/55531/#comment233601>
Please add if (LOG.isDebugEnabled())
server-api/src/main/java/org/apache/atlas/RequestContextV1.java (line 47)
<https://reviews.apache.org/r/55531/#comment233591>
Consider setting this to final and initialize in the constructor.
server-api/src/main/java/org/apache/atlas/RequestContextV1.java (line 59)
<https://reviews.apache.org/r/55531/#comment233590>
since CURRENT_CONTEXT is a ThreadLocal<>, synchronized() usage is not necessary. Please review.
A simpler impl could be:
public static RequestContextV1 get() {
RequestContextV1 ret = CURRENT_CONTEXT.get();
if (ret == null) {
ret = new RequestContextV1();
CURRENT_CONTEXT.set(ret);
}
return ret;
}
server-api/src/main/java/org/apache/atlas/RequestContextV1.java (line 106)
<https://reviews.apache.org/r/55531/#comment233594>
Any reason to create a new list here, instead of simply returning 'createdEntityIds'? The return can be changed to Collection<String>. Please review other methods here for this suggestion.
- Madhan Neethiraj
On Jan. 19, 2017, 6:22 a.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55531/
> -----------------------------------------------------------
>
> (Updated Jan. 19, 2017, 6:22 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1467
> https://issues.apache.org/jira/browse/ATLAS-1467
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Implemented
>
> 1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
> 2. Create with new APIs and POJOs ( no change in graph model )
> 3. Full Update with new APIs and POJOs ( no change in graph model )
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 6770c41
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasEntityDef.java 85e9d77
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
> intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
> intg/src/test/java/org/apache/atlas/type/TestAtlasTypeRegistry.java 9429c07
> repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 2be9a2d
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
> repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 5259249
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
> repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java c7c3286
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
> server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java c4be236
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java 3565ab3
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
> webapp/src/main/java/org/apache/atlas/web/util/LineageUtils.java 54ca236
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 265b650
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java 2a75773
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java dcb1264
> webapp/src/test/java/org/apache/atlas/web/resources/EntityDiscoveryJerseyResourceIT.java 2ade5b0
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d7702e2
>
> Diff: https://reviews.apache.org/r/55531/diff/
>
>
> Testing
> -------
>
> UTs for create
>
> TODO:
> Update testing pending
> Error handling for unresolved references
> UTS for discovery
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/#review162248
-----------------------------------------------------------
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java (line 516)
<https://reviews.apache.org/r/55531/#comment233575>
Please ignore this comment. I noticed that during AtlasAttribute() construction, AtlasStructType is not fully initialized.
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java (line 65)
<https://reviews.apache.org/r/55531/#comment233573>
"instamce" ==> "instance"
intg/src/main/java/org/apache/atlas/model/typedef/AtlasEntityDef.java (line 23)
<https://reviews.apache.org/r/55531/#comment233574>
There is no change in this file except for this import. Please revert this file.
- Madhan Neethiraj
On Jan. 19, 2017, 6:22 a.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55531/
> -----------------------------------------------------------
>
> (Updated Jan. 19, 2017, 6:22 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1467
> https://issues.apache.org/jira/browse/ATLAS-1467
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Implemented
>
> 1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
> 2. Create with new APIs and POJOs ( no change in graph model )
> 3. Full Update with new APIs and POJOs ( no change in graph model )
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 6770c41
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasEntityDef.java 85e9d77
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
> intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
> intg/src/test/java/org/apache/atlas/type/TestAtlasTypeRegistry.java 9429c07
> repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 2be9a2d
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
> repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 5259249
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
> repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java c7c3286
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
> server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java c4be236
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java 3565ab3
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
> webapp/src/main/java/org/apache/atlas/web/util/LineageUtils.java 54ca236
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 265b650
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java 2a75773
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java dcb1264
> webapp/src/test/java/org/apache/atlas/web/resources/EntityDiscoveryJerseyResourceIT.java 2ade5b0
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d7702e2
>
> Diff: https://reviews.apache.org/r/55531/diff/
>
>
> Testing
> -------
>
> UTs for create
>
> TODO:
> Update testing pending
> Error handling for unresolved references
> UTS for discovery
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Suma Shivaprasad <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/
-----------------------------------------------------------
(Updated Jan. 19, 2017, 6:22 a.m.)
Review request for atlas.
Changes
-------
Fixed review comments and update test
Bugs: ATLAS-1467
https://issues.apache.org/jira/browse/ATLAS-1467
Repository: atlas
Description
-------
Implemented
1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
2. Create with new APIs and POJOs ( no change in graph model )
3. Full Update with new APIs and POJOs ( no change in graph model )
Diffs (updated)
-----
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 6770c41
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
intg/src/main/java/org/apache/atlas/model/typedef/AtlasEntityDef.java 85e9d77
intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
intg/src/test/java/org/apache/atlas/type/TestAtlasTypeRegistry.java 9429c07
repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 2be9a2d
repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 5259249
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java c7c3286
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java c4be236
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java 3565ab3
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
webapp/src/main/java/org/apache/atlas/web/util/LineageUtils.java 54ca236
webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 265b650
webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java 2a75773
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java dcb1264
webapp/src/test/java/org/apache/atlas/web/resources/EntityDiscoveryJerseyResourceIT.java 2ade5b0
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d7702e2
Diff: https://reviews.apache.org/r/55531/diff/
Testing
-------
UTs for create
TODO:
Update testing pending
Error handling for unresolved references
UTS for discovery
Thanks,
Suma Shivaprasad
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Suma Shivaprasad <su...@gmail.com>.
> On Jan. 15, 2017, 7:53 p.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java, line 69
> > <https://reviews.apache.org/r/55531/diff/1/?file=1605382#file1605382line69>
> >
> > rename getPropertyKey() ==> getTypeDefPropertyKey()?
Renamed it for better understanbdibiltity that it is only for TypeDef property keys
- Suma
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/#review161657
-----------------------------------------------------------
On Jan. 19, 2017, 10:13 p.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55531/
> -----------------------------------------------------------
>
> (Updated Jan. 19, 2017, 10:13 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1467
> https://issues.apache.org/jira/browse/ATLAS-1467
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Implemented
>
> 1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
> 2. Create with new APIs and POJOs ( no change in graph model )
> 3. Full Update with new APIs and POJOs ( no change in graph model )
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java a6438ed
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
> intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
> intg/src/test/java/org/apache/atlas/type/TestAtlasTypeRegistry.java 9429c07
> repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 2be9a2d
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
> repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 5259249
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
> repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java c7c3286
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
> server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java c4be236
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java 3565ab3
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
> webapp/src/main/java/org/apache/atlas/web/util/LineageUtils.java 54ca236
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 265b650
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java 2a75773
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 44d8a11
> webapp/src/test/java/org/apache/atlas/web/resources/EntityDiscoveryJerseyResourceIT.java 2ade5b0
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d7702e2
>
> Diff: https://reviews.apache.org/r/55531/diff/
>
>
> Testing
> -------
>
> UTs for create
>
> TODO:
> Update testing pending
> Error handling for unresolved references
> UTS for discovery
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/#review161657
-----------------------------------------------------------
I am yet to complete the review; here are few comments so far.
intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java (line 54)
<https://reviews.apache.org/r/55531/#comment232943>
Consider the following alternative:
- Introduce class "org.apache.atlas.type.AtlasAttribute", which encapsulates AtlasAttributeDef:
public AtlasAttribute(AtasStructDef structDef, AtlasAttributeDef attrDef) {
this.structDef = structDef;
this.attrDef = attrDef;
this.qualifiedAttributeName = structDef.getName() + "." + attrDef.getName();
}
public AtlasAttributeDef getAttributeDef() { return attrDef; }
public String getQualifiedAttributeName() { return qualifiedAttributeName; }
- Change type of "allAttributeDef" as below:
Map<String, AtlasAttribute> allAttributeDefs = Collections.emptyMap();
This will be a good place to keep attribute specific additional functionalties (like validation, constraints, ..)
- Remove "allQualifiedAttributeNames"
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java (line 176)
<https://reviews.apache.org/r/55531/#comment232946>
When obj is of type AtlasEntity or AtlasObjectId, consider adding validation of the "typeName" - to ensure that the instance type matches the AtlasEntityType.
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java (line 178)
<https://reviews.apache.org/r/55531/#comment232944>
"obj instanceof AtlasObjectId" - consider moving this to before line #172 "for (AtlasEntityType superType : superTypes) {".
intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java (line 186)
<https://reviews.apache.org/r/55531/#comment232945>
What is the usecase to treat a string (guid?) as an entity? Shouldn't AtlasObjectId be used?
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java (line 462)
<https://reviews.apache.org/r/55531/#comment232926>
this requires that "." is not a valid character in attribute name. Please add validation during type-creation.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java (line 69)
<https://reviews.apache.org/r/55531/#comment232947>
rename getPropertyKey() ==> getTypeDefPropertyKey()?
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java (line 97)
<https://reviews.apache.org/r/55531/#comment232948>
- consider using "AtlasStructType" for the first parameter and avoid "throws AtlasBaseException
- would "fromType" be more appropriate name for "parentType"? Please review.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java (line 147)
<https://reviews.apache.org/r/55531/#comment232949>
Consider renaming "element" to "vertex"
- Madhan Neethiraj
On Jan. 14, 2017, 3:57 a.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55531/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2017, 3:57 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1467
> https://issues.apache.org/jira/browse/ATLAS-1467
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Implemented
>
> 1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
> 2. Create with new APIs and POJOs ( no change in graph model )
> 3. Full Update with new APIs and POJOs ( no change in graph model )
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 6770c41
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasEntityDef.java 85e9d77
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
> intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
> repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java be9fed2
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
> repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java bcdf344
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
> repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 7444bf3
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
> server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
>
> Diff: https://reviews.apache.org/r/55531/diff/
>
>
> Testing
> -------
>
> UTs for create
>
> TODO:
> Update testing pending
> Error handling for unresolved references
> UTS for discovery
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Suma Shivaprasad <su...@gmail.com>.
> On Jan. 16, 2017, 2:57 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java, line 107
> > <https://reviews.apache.org/r/55531/diff/1/?file=1605378#file1605378line107>
> >
> > in what cases would deleteEdgeReferences() return false? And why these edges should be retained (i.e. treated as additionalElements)?
It returns in softDelete case. For hard deletes, return is true
- Suma
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/#review161668
-----------------------------------------------------------
On Jan. 19, 2017, 6:22 a.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55531/
> -----------------------------------------------------------
>
> (Updated Jan. 19, 2017, 6:22 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1467
> https://issues.apache.org/jira/browse/ATLAS-1467
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Implemented
>
> 1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
> 2. Create with new APIs and POJOs ( no change in graph model )
> 3. Full Update with new APIs and POJOs ( no change in graph model )
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 6770c41
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasEntityDef.java 85e9d77
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
> intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
> intg/src/test/java/org/apache/atlas/type/TestAtlasTypeRegistry.java 9429c07
> repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 2be9a2d
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
> repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 5259249
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
> repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java c7c3286
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
> server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java c4be236
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java 3565ab3
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
> webapp/src/main/java/org/apache/atlas/web/util/LineageUtils.java 54ca236
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 265b650
> webapp/src/test/java/org/apache/atlas/web/adapters/TestEntityREST.java 2a75773
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java dcb1264
> webapp/src/test/java/org/apache/atlas/web/resources/EntityDiscoveryJerseyResourceIT.java 2ade5b0
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d7702e2
>
> Diff: https://reviews.apache.org/r/55531/diff/
>
>
> Testing
> -------
>
> UTs for create
>
> TODO:
> Update testing pending
> Error handling for unresolved references
> UTS for discovery
>
>
> Thanks,
>
> Suma Shivaprasad
>
>
Re: Review Request 55531: ATLAS-1467 Create/Full Update implementation
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55531/#review161668
-----------------------------------------------------------
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java (line 62)
<https://reviews.apache.org/r/55531/#comment232950>
repositoryResolvedReferences seems to be populated with "guid" as the key. Is parameter "entity" expected to be an entity guid?
Please review.
repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java (line 106)
<https://reviews.apache.org/r/55531/#comment232951>
unresolvedEntityReferences ==> unresolvedIdReferences
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 29)
<https://reviews.apache.org/r/55531/#comment232952>
consider adding "final" qualifier.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 42)
<https://reviews.apache.org/r/55531/#comment232956>
add "@Override" annotation.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 58)
<https://reviews.apache.org/r/55531/#comment232958>
isn't this same as the previous line?
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 61)
<https://reviews.apache.org/r/55531/#comment232959>
This would overwrite the previous edge storted in the context. Perhaps we should consider context types for array and maps - like:
GraphArrayMutationContext
GraphMapMutationContext
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 75)
<https://reviews.apache.org/r/55531/#comment232960>
Is removal of any other existing edges relevant only for reference type elements? How about for structs (and classifications)?
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 97)
<https://reviews.apache.org/r/55531/#comment232969>
Consider replacing #97, #98 with the following:
Collection<AtlasEdge> edgesToRemove = CollectionUtils.subtract(currentEntries, newEntries);
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 99)
<https://reviews.apache.org/r/55531/#comment232962>
Consider moving this to inside the 'if' block at line #102.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 107)
<https://reviews.apache.org/r/55531/#comment232961>
in what cases would deleteEdgeReferences() return false? And why these edges should be retained (i.e. treated as additionalElements)?
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 112)
<https://reviews.apache.org/r/55531/#comment232963>
Consider moving this to line #111 i.e. inside the previous 'if' block.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 115)
<https://reviews.apache.org/r/55531/#comment232964>
Consider returning 'null' (and update the caller to handle null) or Collections.emptyList().
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java (line 129)
<https://reviews.apache.org/r/55531/#comment232970>
For better readability, consider changing the name (and signature) of the method to:
- getEdgeAt(List<Object> elements, int index, AtlasArrayType arrayType)
- newElements parameter seems to be used only in debug log
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java (line 71)
<https://reviews.apache.org/r/55531/#comment232953>
please add "@Override" annotation.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java (line 23)
<https://reviews.apache.org/r/55531/#comment232954>
Consider using parentType.getStructDef(), instead of using another member "structDef" here?
In fact, it might be useful to replace parentType/structDef/attributeDef/attrType with a reference to AtlasAttribute (please see earlier comment).
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java (line 55)
<https://reviews.apache.org/r/55531/#comment232955>
"currentEdge" - the attribute name doesn't sound good. Please review.
- Madhan Neethiraj
On Jan. 14, 2017, 3:57 a.m., Suma Shivaprasad wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55531/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2017, 3:57 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1467
> https://issues.apache.org/jira/browse/ATLAS-1467
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Implemented
>
> 1. Entity graph discovery - This differs from current implementation in that it collects all references first and resolves them later . The current implemenation tries to resolve references at the same time resulting in issues when entities have cross references to each other . Also it handles resolving a refernce by its unique attribute (by supplying an AtlasEntity instead of an Id)
> 2. Create with new APIs and POJOs ( no change in graph model )
> 3. Full Update with new APIs and POJOs ( no change in graph model )
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 6770c41
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 2ad0f76
> intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java e7b70aa
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 3501c90
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasEntityDef.java 85e9d77
> intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java aee26ef
> intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 8772720
> intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 3625f72
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java e20af76
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 53b109c
> repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java 54dda50
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java be9fed2
> repository/src/main/java/org/apache/atlas/discovery/EntityLineageService.java 45e2dd2
> repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java bcdf344
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java f17b816
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscovery.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityGraphDiscoveryContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/EntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java e731c11
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEnumDefStoreV1.java fccbeba
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 18b3b85
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java e780dc1
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/IDBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/InstanceGraphMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/util/AtlasRepositoryConfiguration.java ea0e670
> repository/src/test/java/org/apache/atlas/lineage/EntityLineageServiceTest.java b1dac9d
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 7444bf3
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java PRE-CREATION
> server-api/src/main/java/org/apache/atlas/RequestContextV1.java PRE-CREATION
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java ad16be7
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java f6acd07
>
> Diff: https://reviews.apache.org/r/55531/diff/
>
>
> Testing
> -------
>
> UTs for create
>
> TODO:
> Update testing pending
> Error handling for unresolved references
> UTS for discovery
>
>
> Thanks,
>
> Suma Shivaprasad
>
>