You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Jeff Hagelberg <jn...@us.ibm.com> on 2017/02/04 02:49:20 UTC
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 4, 2017, 2:49 a.m.)
Review request for atlas and David Kantor.
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description
-------
Adds guidAssignments section to REST API output when creating / updating entities.
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). Prior to this change, the call to createEntity() to create the entity with an invalid was returning null. This was not happening because the date was invalid, but instead because an exception (org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>)) was being thrown during the processing of the call. This behavior changed because the "db" attribute in hive_table_v2 was changed to be a composite reference as part of these changes. Now, the exception is no longer being thrown and so null is not being returned. The test was updated to reflect this.
Diffs
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 8d1bfa7078884d75ef2fb94e65e6dfd41cfeca25
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java c9b6e9712a963a64ec5b91cef657d3b70e211ede
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 7476c4287b74e018999b76e5c57f6d837ab43422
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/util/RestUtils.java f928a063407fbf783e41828e4662802de5383d64
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7368c723c4a0cf4f882440bf05c4b8624e800797
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 510776722ea859d8d7d33d3bf22e76f989138457
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java aa209f98ed8151736f173a52f3e62c0eb55254c9
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d9bffe963d421f77e25ee5ec2cc51b496281eec7
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
> On Feb. 6, 2017, 6:59 p.m., Apoorv Naik wrote:
> > intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java, line 44
> > <https://reviews.apache.org/r/56283/diff/4/?file=1625039#file1625039line44>
> >
> > Use Jackson annotations to control json serialization/de-serialization instead of Gson. The model classes should be pure POJOs
I've addded the jackson annotations. The gson-based serialization stuff is needed since this class is used in both V1 and V2.
- Jeff
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/#review164386
-----------------------------------------------------------
On Feb. 7, 2017, 4:17 a.m., Jeff Hagelberg wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56283/
> -----------------------------------------------------------
>
> (Updated Feb. 7, 2017, 4:17 a.m.)
>
>
> Review request for atlas and David Kantor.
>
>
> Bugs: ATLAS-746
> https://issues.apache.org/jira/browse/ATLAS-746
>
>
> Repository: atlas
>
>
> Description
> -------
>
> When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
>
> This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
>
> Example:
>
> {
> ���"entitiesMutated":{
> � \u2026
> ���},
> ���"guidAssignments":{
> ������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
> ������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
> ������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
> ������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
> ������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
> ������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
> ������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
> ������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
> ������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
> ������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
> ������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
> ������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
> ������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
> ������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
> ������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
> ������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
> ������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
> ������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
> ������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
> ������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
> ������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
> ���}
> }
>
>
> I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
>
> One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
>
> org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
>
> This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
>
> As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is now able to be created successsfully. I've updated the test accordingly.
>
>
> Diffs
> -----
>
> catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
> client/src/main/java/org/apache/atlas/AtlasBaseClient.java 9b699914bd4870683f271c2bab0e3f87ef76fb9a
> client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
> client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 2b3669dde8a67e054675ea44a90c281ffecc32af
> client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 874a43ab9730b23958022e4e64c346e0925ef5cd
> intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java d48c87e2630f81125192ae89bce2b32c2b8ff5ff
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java 88197aca7966cb1a0d8f5b33bedeb9ff27689b52
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
> repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
> repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
> repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
> repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
> repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
> server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
> typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
> webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 1559eb91d89e610f96f0c2ddbf6ad1ce7095773f
> webapp/src/main/java/org/apache/atlas/util/RestUtils.java cecf636a0b3e3c9c9806497768c0986d773f1c5b
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7f5a05696f415eb56ffa90dedc3d6708a921b046
> webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java af0377e9f9c52a52b4f3ebc3f6fa13762f5569d3
> webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 9c0ccf66671ec09524b34b29bb563959db0659ef
> webapp/src/test/java/org/apache/atlas/examples/QuickStartV2IT.java 952813917cd7a519bc3e6e2ae7f30c988ad9db91
> webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
> webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java 29a6d4913e3742ea54cbd4d23ca8ba439c48b23a
>
> Diff: https://reviews.apache.org/r/56283/diff/
>
>
> Testing
> -------
>
> Ran full build, no regressions found
>
>
> Thanks,
>
> Jeff Hagelberg
>
>
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Apoorv Naik <na...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/#review164386
-----------------------------------------------------------
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java (line 44)
<https://reviews.apache.org/r/56283/#comment236077>
Use Jackson annotations to control json serialization/de-serialization instead of Gson. The model classes should be pure POJOs
- Apoorv Naik
On Feb. 6, 2017, 5:37 p.m., Jeff Hagelberg wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56283/
> -----------------------------------------------------------
>
> (Updated Feb. 6, 2017, 5:37 p.m.)
>
>
> Review request for atlas and David Kantor.
>
>
> Bugs: ATLAS-746
> https://issues.apache.org/jira/browse/ATLAS-746
>
>
> Repository: atlas
>
>
> Description
> -------
>
> When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
>
> This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
>
> Example:
>
> {
> ���"entitiesMutated":{
> � \u2026
> ���},
> ���"guidAssignments":{
> ������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
> ������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
> ������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
> ������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
> ������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
> ������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
> ������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
> ������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
> ������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
> ������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
> ������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
> ������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
> ������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
> ������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
> ������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
> ������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
> ������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
> ������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
> ������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
> ������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
> ������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
> ���}
> }
>
>
> I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
>
> One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
>
> org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
>
> This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
>
> As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is now able to be created successsfully. I've updated the test accordingly.
>
>
> Diffs
> -----
>
> catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
> client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
> client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 8d1bfa7078884d75ef2fb94e65e6dfd41cfeca25
> client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java c9b6e9712a963a64ec5b91cef657d3b70e211ede
> intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 7476c4287b74e018999b76e5c57f6d837ab43422
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java 88197aca7966cb1a0d8f5b33bedeb9ff27689b52
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
> repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
> repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
> repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
> repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
> repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
> server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
> typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
> webapp/src/main/java/org/apache/atlas/util/RestUtils.java f928a063407fbf783e41828e4662802de5383d64
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7368c723c4a0cf4f882440bf05c4b8624e800797
> webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 510776722ea859d8d7d33d3bf22e76f989138457
> webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java aa209f98ed8151736f173a52f3e62c0eb55254c9
> webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
> webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d9bffe963d421f77e25ee5ec2cc51b496281eec7
>
> Diff: https://reviews.apache.org/r/56283/diff/
>
>
> Testing
> -------
>
> Ran full build, no regressions found
>
>
> Thanks,
>
> Jeff Hagelberg
>
>
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 7, 2017, 7:42 p.m.)
Review request for atlas and David Kantor.
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description
-------
When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is now able to be created successsfully. I've updated the test accordingly.
Diffs (updated)
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasBaseClient.java 9b699914bd4870683f271c2bab0e3f87ef76fb9a
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 2b3669dde8a67e054675ea44a90c281ffecc32af
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 874a43ab9730b23958022e4e64c346e0925ef5cd
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java d48c87e2630f81125192ae89bce2b32c2b8ff5ff
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java 88197aca7966cb1a0d8f5b33bedeb9ff27689b52
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 1559eb91d89e610f96f0c2ddbf6ad1ce7095773f
webapp/src/main/java/org/apache/atlas/util/RestUtils.java cecf636a0b3e3c9c9806497768c0986d773f1c5b
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7f5a05696f415eb56ffa90dedc3d6708a921b046
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java af0377e9f9c52a52b4f3ebc3f6fa13762f5569d3
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 9c0ccf66671ec09524b34b29bb563959db0659ef
webapp/src/test/java/org/apache/atlas/examples/QuickStartV2IT.java 952813917cd7a519bc3e6e2ae7f30c988ad9db91
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java 29a6d4913e3742ea54cbd4d23ca8ba439c48b23a
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 7, 2017, 4:08 p.m.)
Review request for atlas and David Kantor.
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description
-------
When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is now able to be created successsfully. I've updated the test accordingly.
Diffs (updated)
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasBaseClient.java 9b699914bd4870683f271c2bab0e3f87ef76fb9a
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 2b3669dde8a67e054675ea44a90c281ffecc32af
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 874a43ab9730b23958022e4e64c346e0925ef5cd
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java d48c87e2630f81125192ae89bce2b32c2b8ff5ff
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java 88197aca7966cb1a0d8f5b33bedeb9ff27689b52
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 1559eb91d89e610f96f0c2ddbf6ad1ce7095773f
webapp/src/main/java/org/apache/atlas/util/RestUtils.java cecf636a0b3e3c9c9806497768c0986d773f1c5b
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7f5a05696f415eb56ffa90dedc3d6708a921b046
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java af0377e9f9c52a52b4f3ebc3f6fa13762f5569d3
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 9c0ccf66671ec09524b34b29bb563959db0659ef
webapp/src/test/java/org/apache/atlas/examples/QuickStartV2IT.java 952813917cd7a519bc3e6e2ae7f30c988ad9db91
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java 29a6d4913e3742ea54cbd4d23ca8ba439c48b23a
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 7, 2017, 2:45 p.m.)
Review request for atlas and David Kantor.
Changes
-------
cleaned up trailing whitespace in new changes.
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description
-------
When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is now able to be created successsfully. I've updated the test accordingly.
Diffs (updated)
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasBaseClient.java 9b699914bd4870683f271c2bab0e3f87ef76fb9a
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 2b3669dde8a67e054675ea44a90c281ffecc32af
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 874a43ab9730b23958022e4e64c346e0925ef5cd
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java d48c87e2630f81125192ae89bce2b32c2b8ff5ff
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java 88197aca7966cb1a0d8f5b33bedeb9ff27689b52
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 1559eb91d89e610f96f0c2ddbf6ad1ce7095773f
webapp/src/main/java/org/apache/atlas/util/RestUtils.java cecf636a0b3e3c9c9806497768c0986d773f1c5b
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7f5a05696f415eb56ffa90dedc3d6708a921b046
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java af0377e9f9c52a52b4f3ebc3f6fa13762f5569d3
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 9c0ccf66671ec09524b34b29bb563959db0659ef
webapp/src/test/java/org/apache/atlas/examples/QuickStartV2IT.java 952813917cd7a519bc3e6e2ae7f30c988ad9db91
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java 29a6d4913e3742ea54cbd4d23ca8ba439c48b23a
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 7, 2017, 2:38 p.m.)
Review request for atlas and David Kantor.
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description
-------
When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is now able to be created successsfully. I've updated the test accordingly.
Diffs (updated)
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasBaseClient.java 9b699914bd4870683f271c2bab0e3f87ef76fb9a
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 2b3669dde8a67e054675ea44a90c281ffecc32af
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 874a43ab9730b23958022e4e64c346e0925ef5cd
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java d48c87e2630f81125192ae89bce2b32c2b8ff5ff
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java 88197aca7966cb1a0d8f5b33bedeb9ff27689b52
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 1559eb91d89e610f96f0c2ddbf6ad1ce7095773f
webapp/src/main/java/org/apache/atlas/util/RestUtils.java cecf636a0b3e3c9c9806497768c0986d773f1c5b
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7f5a05696f415eb56ffa90dedc3d6708a921b046
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java af0377e9f9c52a52b4f3ebc3f6fa13762f5569d3
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 9c0ccf66671ec09524b34b29bb563959db0659ef
webapp/src/test/java/org/apache/atlas/examples/QuickStartV2IT.java 952813917cd7a519bc3e6e2ae7f30c988ad9db91
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java 29a6d4913e3742ea54cbd4d23ca8ba439c48b23a
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 7, 2017, 4:17 a.m.)
Review request for atlas and David Kantor.
Changes
-------
Rebased changes, fixed issues with EntityV2JerseyResourceIT
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description
-------
When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is now able to be created successsfully. I've updated the test accordingly.
Diffs (updated)
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasBaseClient.java 9b699914bd4870683f271c2bab0e3f87ef76fb9a
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 2b3669dde8a67e054675ea44a90c281ffecc32af
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 874a43ab9730b23958022e4e64c346e0925ef5cd
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java d48c87e2630f81125192ae89bce2b32c2b8ff5ff
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java 88197aca7966cb1a0d8f5b33bedeb9ff27689b52
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 1559eb91d89e610f96f0c2ddbf6ad1ce7095773f
webapp/src/main/java/org/apache/atlas/util/RestUtils.java cecf636a0b3e3c9c9806497768c0986d773f1c5b
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7f5a05696f415eb56ffa90dedc3d6708a921b046
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java af0377e9f9c52a52b4f3ebc3f6fa13762f5569d3
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 9c0ccf66671ec09524b34b29bb563959db0659ef
webapp/src/test/java/org/apache/atlas/examples/QuickStartV2IT.java 952813917cd7a519bc3e6e2ae7f30c988ad9db91
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java 29a6d4913e3742ea54cbd4d23ca8ba439c48b23a
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 6, 2017, 5:37 p.m.)
Review request for atlas and David Kantor.
Changes
-------
Addressed review comments, added reference to JIRA created for issues found in RestUtilsTest.
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description (updated)
-------
When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is now able to be created successsfully. I've updated the test accordingly.
Diffs (updated)
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 8d1bfa7078884d75ef2fb94e65e6dfd41cfeca25
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java c9b6e9712a963a64ec5b91cef657d3b70e211ede
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 7476c4287b74e018999b76e5c57f6d837ab43422
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java 88197aca7966cb1a0d8f5b33bedeb9ff27689b52
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/util/RestUtils.java f928a063407fbf783e41828e4662802de5383d64
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7368c723c4a0cf4f882440bf05c4b8624e800797
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 510776722ea859d8d7d33d3bf22e76f989138457
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java aa209f98ed8151736f173a52f3e62c0eb55254c9
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d9bffe963d421f77e25ee5ec2cc51b496281eec7
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 6, 2017, 5 p.m.)
Review request for atlas and David Kantor.
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description
-------
When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is able to be created successsfully. I've updated the test accordingly.
Diffs (updated)
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 8d1bfa7078884d75ef2fb94e65e6dfd41cfeca25
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java c9b6e9712a963a64ec5b91cef657d3b70e211ede
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 7476c4287b74e018999b76e5c57f6d837ab43422
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java 88197aca7966cb1a0d8f5b33bedeb9ff27689b52
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/util/RestUtils.java f928a063407fbf783e41828e4662802de5383d64
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7368c723c4a0cf4f882440bf05c4b8624e800797
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 510776722ea859d8d7d33d3bf22e76f989138457
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java aa209f98ed8151736f173a52f3e62c0eb55254c9
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d9bffe963d421f77e25ee5ec2cc51b496281eec7
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by David Kantor <dk...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/#review164355
-----------------------------------------------------------
Fix it, then Ship it!
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java (line 39)
<https://reviews.apache.org/r/56283/#comment236038>
Outdated javadoc, refers to skeleton objects which are not in this implementation.
repository/src/test/java/org/apache/atlas/TestUtils.java (line 714)
<https://reviews.apache.org/r/56283/#comment236039>
Add javadoc which explains what these methods are doing.
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java (line 1)
<https://reviews.apache.org/r/56283/#comment236040>
Needs Apache header. Remove extra whitespace.
- David Kantor
On Feb. 6, 2017, 3:46 p.m., Jeff Hagelberg wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56283/
> -----------------------------------------------------------
>
> (Updated Feb. 6, 2017, 3:46 p.m.)
>
>
> Review request for atlas and David Kantor.
>
>
> Bugs: ATLAS-746
> https://issues.apache.org/jira/browse/ATLAS-746
>
>
> Repository: atlas
>
>
> Description
> -------
>
> When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
>
> This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
>
> Example:
>
> {
> ���"entitiesMutated":{
> � \u2026
> ���},
> ���"guidAssignments":{
> ������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
> ������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
> ������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
> ������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
> ������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
> ������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
> ������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
> ������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
> ������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
> ������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
> ������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
> ������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
> ������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
> ������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
> ������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
> ������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
> ������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
> ������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
> ������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
> ������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
> ������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
> ���}
> }
>
>
> I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
>
> One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
>
> org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
>
> This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
>
> As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is able to be created successsfully. I've updated the test accordingly.
>
>
> Diffs
> -----
>
> catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
> client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
> client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 8d1bfa7078884d75ef2fb94e65e6dfd41cfeca25
> client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java c9b6e9712a963a64ec5b91cef657d3b70e211ede
> intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 7476c4287b74e018999b76e5c57f6d837ab43422
> repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
> repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
> repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
> repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
> repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
> repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
> server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
> typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
> webapp/src/main/java/org/apache/atlas/util/RestUtils.java f928a063407fbf783e41828e4662802de5383d64
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7368c723c4a0cf4f882440bf05c4b8624e800797
> webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
> webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 510776722ea859d8d7d33d3bf22e76f989138457
> webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java aa209f98ed8151736f173a52f3e62c0eb55254c9
> webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
> webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
> webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
> webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d9bffe963d421f77e25ee5ec2cc51b496281eec7
>
> Diff: https://reviews.apache.org/r/56283/diff/
>
>
> Testing
> -------
>
> Ran full build, no regressions found
>
>
> Thanks,
>
> Jeff Hagelberg
>
>
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 6, 2017, 3:46 p.m.)
Review request for atlas and David Kantor.
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description (updated)
-------
When a an entity is being created that contains nested entities, it is not currently possible to determine what guids were assigned to the nested entities. This is because the result going back to the caller is simply a flat list of the guids of the entities that were created. We get back the guids, but it is impossible to know which nested entities they refer to without executing an additional query.
This task addresses this issue by adding a "guidAssignments" section to REST API output when creating / updating entities. This has a map from the "unassigned" guids in the original request to the guids that were assigned during the entity creation. This makes it easy to determine what the guid is for all the entities that were created, regardlesss of how deeply nested they are.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). This test, on the surface, looked like it was testing that creating an entity with a date in an invalid format would fail. In reality, though, the invalid date was being nulled out and ignored. The failure was actually happening after that point, because the database reference ("db") could not be resolved. It was failing with this exception:
org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>))
This was a direct result of "db" being a non-composite attribute : ClassType.convert() replaces non-composite reference attributes in an entity being created with their Id.
As part of the testing for these changes, I changed the "db" attribute to be a composite attribute. As a result, the entity is able to be created successsfully. I've updated the test accordingly.
Diffs
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 8d1bfa7078884d75ef2fb94e65e6dfd41cfeca25
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java c9b6e9712a963a64ec5b91cef657d3b70e211ede
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 7476c4287b74e018999b76e5c57f6d837ab43422
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/util/RestUtils.java f928a063407fbf783e41828e4662802de5383d64
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7368c723c4a0cf4f882440bf05c4b8624e800797
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 510776722ea859d8d7d33d3bf22e76f989138457
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java aa209f98ed8151736f173a52f3e62c0eb55254c9
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d9bffe963d421f77e25ee5ec2cc51b496281eec7
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg
Re: Review Request 56283: ATLAS-746: After updating a set of entities,
response contains only the first entity definition
Posted by Jeff Hagelberg <jn...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56283/
-----------------------------------------------------------
(Updated Feb. 6, 2017, 2:47 p.m.)
Review request for atlas and David Kantor.
Bugs: ATLAS-746
https://issues.apache.org/jira/browse/ATLAS-746
Repository: atlas
Description
-------
Adds guidAssignments section to REST API output when creating / updating entities.
I also fixed an issue preventing nested entities from being created. The issue there was that the mapping composite attribute representation in the old and new format was inconsistent. I've changed the code so that an attribute is treated as composite if it is either a "mappedFromRef" attribute or it has a foreign key with "update"as the onDelete action. There were two competing ways of determining this.
Example:
{
���"entitiesMutated":{
� \u2026
���},
���"guidAssignments":{
������"-267745036104320":"27423408-4619-4736-9b99-b9f2bb5783b7",
������"-267745036104310":"41628d3d-6da0-4d79-9f6b-863c6631d31f",
������"-267745036104311":"b75acf2d-0523-423e-ac1e-4721b2322b91",
������"-267745036104300":"dbbd27b1-048a-447c-82dd-7ccff717bd4a",
������"-267745036104309":"cc12c71f-6d30-499e-8dab-f243b8547f42",
������"-267745036104316":"f5149832-c3ef-47ae-a982-a29b365e0199",
������"-267745036104305":"fcdd4076-c3e6-4a4c-ac56-c62599ae79c9",
������"-267745036104306":"7cba46ff-52ca-49e6-8327-ab5f7892deba",
������"-267745036104317":"b33aea18-365a-4017-9659-a084593dd47e",
������"-267745036104318":"d1c36c36-0968-4758-b2f9-d41ed6097c1b",
������"-267745036104307":"1da32314-7ac2-4d88-97c9-a1f26edb281b",
������"-267745036104319":"50c707ce-53e2-49c6-862b-b79a37102ae5",
������"-267745036104308":"c49655bc-ba54-4e37-94f2-938c9bc565a5",
������"-267745036104301":"f6190422-09f7-4f17-bda3-a1b7a5f5415d",
������"-267745036104312":"b6b44a3f-6307-4d67-a777-a29764971dba",
������"-267745036104302":"e3f570b7-9b74-43f4-ba6e-c730d8a630b3",
������"-267745036104313":"5d6b033f-eec3-4bd6-b655-be48042a6530",
������"-267745036104314":"0ca94243-9187-43a1-aaad-ced0ea2a1731",
������"-267745036104303":"56b42729-45af-44e5-98d4-a6a77889340a",
������"-267745036104315":"8061b59b-fa1c-4ca8-8f18-72c3bcc82d73",
������"-267745036104304":"b9b3ecf7-47f6-4887-9cd7-32febf0e2fa2"
���}
}
One quick note about EntityV2JerseyResourceIT.testSubmitEntityWithBadDateFormat(). Prior to this change, the call to createEntity() to create the entity with an invalid was returning null. This was not happening because the date was invalid, but instead because an exception (org.apache.atlas.typesystem.exception.EntityNotFoundException: Could not find vertex for Class Reference (type: hive_db_v2, id: <unassigned>)) was being thrown during the processing of the call. This behavior changed because the "db" attribute in hive_table_v2 was changed to be a composite reference as part of these changes. Now, the exception is no longer being thrown and so null is not being returned. The test was updated to reflect this.
Diffs (updated)
-----
catalog/src/main/java/org/apache/atlas/catalog/DefaultTypeSystem.java 726351aa2037c3ead59eaa3fe1d052cc0889f525
client/src/main/java/org/apache/atlas/AtlasClient.java 154644dd9e637d612960a8c811c54efaf18d3b27
client/src/main/java/org/apache/atlas/AtlasEntitiesClientV2.java 8d1bfa7078884d75ef2fb94e65e6dfd41cfeca25
client/src/main/java/org/apache/atlas/CreateUpdateEntitiesResult.java PRE-CREATION
intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java c9b6e9712a963a64ec5b91cef657d3b70e211ede
intg/src/main/java/org/apache/atlas/model/instance/GuidMapping.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java 1d61ea84f33e8a273229cc077cf1c205d7e29f96
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java 27bf6d7278ac22ae9d1b572141e5a19e1293c48f
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java 1a88251f5c465fca6bd9a3dd28dc9afb3540fe25
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 7476c4287b74e018999b76e5c57f6d837ab43422
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java b14531f0b994674daa72773e65476151215d664b
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java 9714a8b2ba3edee65632352d9059dd94374faaae
repository/src/test/java/org/apache/atlas/TestUtils.java 1d1a5e04210ccc48ec989c514d3796c2d7dc211c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java ce87c9ee5ef45dc8339145e40e9e2a6c37a40e9f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java d6136edc7cf29d648292ce6b852e88081c1c3165
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java 96b017364718312b55715a1056a5e43155c9152a
repository/src/test/java/org/apache/atlas/repository/graph/GraphHelperTest.java f5a6a0519b9153e6edc68ed4502dad2c15a5a1e0
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java a03f965f81dadd932bb12c6c6ab62fc7ebb27322
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java 265b316a8582172b045f4313b63d3f4cc7f1ad72
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java 1ad79b1f3e78f712905d57f58e1d00d59d535b70
server-api/src/main/java/org/apache/atlas/services/MetadataService.java d5d8d9b5d57f4bf7fbf49373337d526d67583e5e
typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 1dcad1408e08ff049444cbbe72668efe9b80e55e
webapp/src/main/java/org/apache/atlas/util/RestUtils.java f928a063407fbf783e41828e4662802de5383d64
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasInstanceRestAdapters.java 7368c723c4a0cf4f882440bf05c4b8624e800797
webapp/src/main/java/org/apache/atlas/web/resources/EntityResource.java 9ca684d956724512f787ff5169eece70197bd0b1
webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 510776722ea859d8d7d33d3bf22e76f989138457
webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java aa209f98ed8151736f173a52f3e62c0eb55254c9
webapp/src/test/java/org/apache/atlas/util/RestUtilsTest.java PRE-CREATION
webapp/src/test/java/org/apache/atlas/web/resources/BaseResourceIT.java 18bbc3bc3e26c66885b652d9d4efdc538c31f0cc
webapp/src/test/java/org/apache/atlas/web/resources/EntityJerseyResourceIT.java 9724c4c4bb2dc6c209cdf643d0eb4eb4856eb981
webapp/src/test/java/org/apache/atlas/web/resources/EntityV2JerseyResourceIT.java d9bffe963d421f77e25ee5ec2cc51b496281eec7
Diff: https://reviews.apache.org/r/56283/diff/
Testing
-------
Ran full build, no regressions found
Thanks,
Jeff Hagelberg