You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by David Kantor <dk...@us.ibm.com> on 2017/02/22 23:48:15 UTC
Review Request 56860: ATLAS-1552 auto update of inverse references in
V2 code path
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/
-----------------------------------------------------------
Review request for atlas.
Bugs: ATLAS-1552
https://issues.apache.org/jira/browse/ATLAS-1552
Repository: atlas
Description
-------
Automatic update of inverse references in V2 (V1?) code path.
Diffs
-----
intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 49d5a080158e3e54fd917a13f743f1cf495e66b0
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
Diff: https://reviews.apache.org/r/56860/diff/
Testing
-------
Ran all unit and integration tests with no regressions.
Thanks,
David Kantor
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166543
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Feb. 22, 2017, 11:48 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2017, 11:48 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 49d5a080158e3e54fd917a13f743f1cf495e66b0
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by David Kantor <dk...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/
-----------------------------------------------------------
(Updated March 1, 2017, 2:18 p.m.)
Review request for atlas.
Changes
-------
Rebase with master
Bugs: ATLAS-1552
https://issues.apache.org/jira/browse/ATLAS-1552
Repository: atlas
Description
-------
Automatic update of inverse references in V2 (V1?) code path.
Diffs (updated)
-----
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java bb7eef88a44c6292a9b51c72dd530dba5ffa274e
intg/src/test/java/org/apache/atlas/TestUtilsV2.java 0b28bcf1ebba5fa6f936964934bf60caee91fa3b
intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java e2b82ccd89eb2053b633f7f070b1744dc5755299
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java aab0d3eeee87e97f36541bf9f935e8410ff87aed
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
Diff: https://reviews.apache.org/r/56860/diff/4/
Changes: https://reviews.apache.org/r/56860/diff/3-4/
Testing
-------
Ran all unit and integration tests with no regressions.
Thanks,
David Kantor
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166719
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Feb. 24, 2017, 3 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2017, 3 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 57ad1063eb0d225102a2b8b72232054f57b9c36b
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by David Kantor <dk...@us.ibm.com>.
> On Feb. 27, 2017, 5:27 p.m., David Radley wrote:
> > I have done a little testing at V2 to model a parent children hierarchy that we would need to define a Glossary (see Jira 1410)
> >
> > I define types EntityA with a children 1 to many optional attribute to EntityB. EntityB has a mantidory pointer to an EntityA parent.
> >
> > EntityA type has
> > "name": "children",
> > "typeName": "array<EntityB>",
> > "cardinality": "SINGLE",
> > "constraintDefs": [
> > {
> > "type": "ownedRef"
> > }
> > ],
> > ],
> > "isIndexable": false,
> > "isOptional": true,
> > "isUnique": false
> > },
> > EntityB type has attributes like this:
> > "name": "parent",
> > "typeName": "EntityA",
> > "cardinality": "SINGLE",
> > "constraintDefs": [
> > {
> > "type": "inverseRef",
> > "params": {
> > "attribute": "children"
> > }
> > }
> > ],
> > "isIndexable": false,
> > "isOptional": false,
> > "isUnique": false
> >
> > I then define
> > -an instance of EntityA
> > -an instance of EntityB with parent pointing to EntityA (I have to supply the parent attribute as it is isOptional false).
> >
> > If I then get (using the API) the EntityA instance it does not have any content in its children. Is this expected? It seems that if I am allowed to do this then the children and parent pointers are out of sync.
> >
> > Am I missing something?
I added a test case that mimics your use case and could not reproduce the issue - when I get the EntityA instance its children attribute contains the EntityB instance, indicating the inverse reference was updated correctly. Perhaps you were running without my changes? I will follow up with you.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166897
-----------------------------------------------------------
On Feb. 24, 2017, 3 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2017, 3 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 57ad1063eb0d225102a2b8b72232054f57b9c36b
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by David Radley <da...@uk.ibm.com>.
> On Feb. 27, 2017, 5:27 p.m., David Radley wrote:
> > I have done a little testing at V2 to model a parent children hierarchy that we would need to define a Glossary (see Jira 1410)
> >
> > I define types EntityA with a children 1 to many optional attribute to EntityB. EntityB has a mantidory pointer to an EntityA parent.
> >
> > EntityA type has
> > "name": "children",
> > "typeName": "array<EntityB>",
> > "cardinality": "SINGLE",
> > "constraintDefs": [
> > {
> > "type": "ownedRef"
> > }
> > ],
> > ],
> > "isIndexable": false,
> > "isOptional": true,
> > "isUnique": false
> > },
> > EntityB type has attributes like this:
> > "name": "parent",
> > "typeName": "EntityA",
> > "cardinality": "SINGLE",
> > "constraintDefs": [
> > {
> > "type": "inverseRef",
> > "params": {
> > "attribute": "children"
> > }
> > }
> > ],
> > "isIndexable": false,
> > "isOptional": false,
> > "isUnique": false
> >
> > I then define
> > -an instance of EntityA
> > -an instance of EntityB with parent pointing to EntityA (I have to supply the parent attribute as it is isOptional false).
> >
> > If I then get (using the API) the EntityA instance it does not have any content in its children. Is this expected? It seems that if I am allowed to do this then the children and parent pointers are out of sync.
> >
> > Am I missing something?
>
> David Kantor wrote:
> I added a test case that mimics your use case and could not reproduce the issue - when I get the EntityA instance its children attribute contains the EntityB instance, indicating the inverse reference was updated correctly. Perhaps you were running without my changes? I will follow up with you.
There was a mistake in my test case - but the API accepted the request - which led to the unexpected behaviour. I have raised ATLAS-1609 to police API content, and to error on unrecognised (e.g. misspelt) request content. The fix looks good.
Ship it.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166897
-----------------------------------------------------------
On Feb. 24, 2017, 3 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2017, 3 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 57ad1063eb0d225102a2b8b72232054f57b9c36b
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/56860/diff/3/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by David Radley <da...@uk.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166897
-----------------------------------------------------------
I have done a little testing at V2 to model a parent children hierarchy that we would need to define a Glossary (see Jira 1410)
I define types EntityA with a children 1 to many optional attribute to EntityB. EntityB has a mantidory pointer to an EntityA parent.
EntityA type has
"name": "children",
"typeName": "array<EntityB>",
"cardinality": "SINGLE",
"constraintDefs": [
{
"type": "ownedRef"
}
],
],
"isIndexable": false,
"isOptional": true,
"isUnique": false
},
EntityB type has attributes like this:
"name": "parent",
"typeName": "EntityA",
"cardinality": "SINGLE",
"constraintDefs": [
{
"type": "inverseRef",
"params": {
"attribute": "children"
}
}
],
"isIndexable": false,
"isOptional": false,
"isUnique": false
I then define
-an instance of EntityA
-an instance of EntityB with parent pointing to EntityA (I have to supply the parent attribute as it is isOptional false).
If I then get (using the API) the EntityA instance it does not have any content in its children. Is this expected? It seems that if I am allowed to do this then the children and parent pointers are out of sync.
Am I missing something?
- David Radley
On Feb. 24, 2017, 3 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2017, 3 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 57ad1063eb0d225102a2b8b72232054f57b9c36b
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by David Kantor <dk...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/
-----------------------------------------------------------
(Updated Feb. 24, 2017, 3 p.m.)
Review request for atlas.
Changes
-------
Fix log messages
Bugs: ATLAS-1552
https://issues.apache.org/jira/browse/ATLAS-1552
Repository: atlas
Description
-------
Automatic update of inverse references in V2 (V1?) code path.
Diffs (updated)
-----
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 57ad1063eb0d225102a2b8b72232054f57b9c36b
intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
Diff: https://reviews.apache.org/r/56860/diff/
Testing
-------
Ran all unit and integration tests with no regressions.
Thanks,
David Kantor
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by David Kantor <dk...@us.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/
-----------------------------------------------------------
(Updated Feb. 24, 2017, 2:25 p.m.)
Review request for atlas.
Changes
-------
Address review comments
Bugs: ATLAS-1552
https://issues.apache.org/jira/browse/ATLAS-1552
Repository: atlas
Description
-------
Automatic update of inverse references in V2 (V1?) code path.
Diffs (updated)
-----
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 57ad1063eb0d225102a2b8b72232054f57b9c36b
intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
Diff: https://reviews.apache.org/r/56860/diff/
Testing
-------
Ran all unit and integration tests with no regressions.
Thanks,
David Kantor
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by David Kantor <dk...@us.ibm.com>.
> On Feb. 23, 2017, 7:14 a.m., Madhan Neethiraj wrote:
> > Consider the following scenario:
> > - create hive_table 'employee'
> > - create hive_column 'name'; set attribute 'table' to 'employee' table created above
> > - now 'employee.columns' attribute lists 'name' - good!
> > - create hive_table 'person'
> > - edit the hive_column created above ('name'), to set attribute 'table' to 'person' table
> > - now 'person.columns' attribute lists 'name' - good!
> >
> > However, 'name' attribute is still seen in 'employee' table. This should not be the case, right? Can you please review?
I think what you are seeing in this use case is the result of having soft deletes enabled, and based on my understanding of how soft deletion is implemented, it is the expected behavior. The graph edge which represents the reference from hive_table(employee).columns to hive_column(name) is in fact deleted. With soft deletes enabled, deleting an edge or vertex amounts to setting its __state property to DELETED. However, the edge ID for the deleted edge is *not* deleted from the hive_table(employee).columns property value - see the comments in DeleteHandlerV1.deleteEdgeBetweenVertices() lines 465-467 (which were copied from DeleteHandler.deleteEdgeBetweenVertices() lines 349-351). When soft deletes are enabled, shouldUpdateReverseAttribute is false, so the logic to remove the edge ID for the deleted edge from the reference property value is not executed, and the edge ID for the deleted edge remains in the property. When the entity is retrieved, there is no logic that I could find
which filters out soft deleted edges for a multiplicity-many reference (i.e. an attribute that is an array of CLASS (legacy) or OBJECT_ID_TYPE (new) elements) in either the legacy code path (GraphToTypedInstanceMapper) or in the new code path (EntityGraphRetriever). The specific logic to not delete the edge ID from the vertex property (and soft deletes in general) was implemented by Shwetha Shivalingamurthy. Perhaps she could comment on this.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166483
-----------------------------------------------------------
On Feb. 22, 2017, 11:48 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2017, 11:48 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 49d5a080158e3e54fd917a13f743f1cf495e66b0
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by Madhan Neethiraj <ma...@apache.org>.
> On Feb. 23, 2017, 7:14 a.m., Madhan Neethiraj wrote:
> > Consider the following scenario:
> > - create hive_table 'employee'
> > - create hive_column 'name'; set attribute 'table' to 'employee' table created above
> > - now 'employee.columns' attribute lists 'name' - good!
> > - create hive_table 'person'
> > - edit the hive_column created above ('name'), to set attribute 'table' to 'person' table
> > - now 'person.columns' attribute lists 'name' - good!
> >
> > However, 'name' attribute is still seen in 'employee' table. This should not be the case, right? Can you please review?
>
> David Kantor wrote:
> I think what you are seeing in this use case is the result of having soft deletes enabled, and based on my understanding of how soft deletion is implemented, it is the expected behavior. The graph edge which represents the reference from hive_table(employee).columns to hive_column(name) is in fact deleted. With soft deletes enabled, deleting an edge or vertex amounts to setting its __state property to DELETED. However, the edge ID for the deleted edge is *not* deleted from the hive_table(employee).columns property value - see the comments in DeleteHandlerV1.deleteEdgeBetweenVertices() lines 465-467 (which were copied from DeleteHandler.deleteEdgeBetweenVertices() lines 349-351). When soft deletes are enabled, shouldUpdateReverseAttribute is false, so the logic to remove the edge ID for the deleted edge from the reference property value is not executed, and the edge ID for the deleted edge remains in the property. When the entity is retrieved, there is no logic that I could
find which filters out soft deleted edges for a multiplicity-many reference (i.e. an attribute that is an array of CLASS (legacy) or OBJECT_ID_TYPE (new) elements) in either the legacy code path (GraphToTypedInstanceMapper) or in the new code path (EntityGraphRetriever). The specific logic to not delete the edge ID from the vertex property (and soft deletes in general) was implemented by Shwetha Shivalingamurthy. Perhaps she could comment on this.
Thanks for the details. I think you are right - this seems to be the result of how soft-delete is handled. We will have to think through more on how to handle this state, to avoid potential confusion for users.
Rest of the patch looks good.
- Madhan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166483
-----------------------------------------------------------
On Feb. 22, 2017, 11:48 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2017, 11:48 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 49d5a080158e3e54fd917a13f743f1cf495e66b0
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by David Kantor <dk...@us.ibm.com>.
> On Feb. 23, 2017, 7:14 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java, line 333
> > <https://reviews.apache.org/r/56860/diff/1/?file=1644998#file1644998line333>
> >
> > It might be useful to add a method in AtlasAttribute, to return reverse attribute's AtlasAttribute. This can be initialized in AtlasEntityType.resolveReferencesPhase2().
> >
> > This will help avoid lookups during entity attribute updates.
Good idea, I have applied the suggested changes.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166483
-----------------------------------------------------------
On Feb. 24, 2017, 2:25 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2017, 2:25 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 57ad1063eb0d225102a2b8b72232054f57b9c36b
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 2d71d4135025339cd072982efa1231575bf21035
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java 3f87e695b01a21f0cbd453f6de68269c5fe025aa
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java f1efe46404d71187c2691f9b795916ed6c26ef3e
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166483
-----------------------------------------------------------
Consider the following scenario:
- create hive_table 'employee'
- create hive_column 'name'; set attribute 'table' to 'employee' table created above
- now 'employee.columns' attribute lists 'name' - good!
- create hive_table 'person'
- edit the hive_column created above ('name'), to set attribute 'table' to 'person' table
- now 'person.columns' attribute lists 'name' - good!
However, 'name' attribute is still seen in 'employee' table. This should not be the case, right? Can you please review?
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java (line 333)
<https://reviews.apache.org/r/56860/#comment238450>
It might be useful to add a method in AtlasAttribute, to return reverse attribute's AtlasAttribute. This can be initialized in AtlasEntityType.resolveReferencesPhase2().
This will help avoid lookups during entity attribute updates.
- Madhan Neethiraj
On Feb. 22, 2017, 11:48 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2017, 11:48 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 49d5a080158e3e54fd917a13f743f1cf495e66b0
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by Suma Shivaprasad <su...@gmail.com>.
> On Feb. 23, 2017, 7:23 p.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java, line 330
> > <https://reviews.apache.org/r/56860/diff/1/?file=1644999#file1644999line330>
> >
> > handle map case as well?
>
> David Kantor wrote:
> Maps are specifically not supported for inverse reference updating, in both the legacy code I delivered recently for ATLAS-1551 and these changes - if the inverse reference is a map, what would be used for the key when adding the map entry? The map attribute itself can have a inverse reference, and if that inverse reference is not a map, it will be automatically updated. I will add a test to verify that use case.
Makes sense. Thanks
- Suma
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166552
-----------------------------------------------------------
On Feb. 22, 2017, 11:48 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2017, 11:48 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 49d5a080158e3e54fd917a13f743f1cf495e66b0
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by David Kantor <dk...@us.ibm.com>.
> On Feb. 23, 2017, 7:23 p.m., Suma Shivaprasad wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java, line 330
> > <https://reviews.apache.org/r/56860/diff/1/?file=1644999#file1644999line330>
> >
> > handle map case as well?
Maps are specifically not supported for inverse reference updating, in both the legacy code I delivered recently for ATLAS-1551 and these changes - if the inverse reference is a map, what would be used for the key when adding the map entry? The map attribute itself can have a inverse reference, and if that inverse reference is not a map, it will be automatically updated. I will add a test to verify that use case.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166552
-----------------------------------------------------------
On Feb. 22, 2017, 11:48 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2017, 11:48 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 49d5a080158e3e54fd917a13f743f1cf495e66b0
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>
Re: Review Request 56860: ATLAS-1552 auto update of inverse
references in V2 code path
Posted by Suma Shivaprasad <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56860/#review166552
-----------------------------------------------------------
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java (line 330)
<https://reviews.apache.org/r/56860/#comment238510>
handle map case as well?
- Suma Shivaprasad
On Feb. 22, 2017, 11:48 p.m., David Kantor wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56860/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2017, 11:48 p.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1552
> https://issues.apache.org/jira/browse/ATLAS-1552
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Automatic update of inverse references in V2 (V1?) code path.
>
>
> Diffs
> -----
>
> intg/src/test/java/org/apache/atlas/TestUtilsV2.java 84e17cb364fbf6d07b2211d85e336d8146fdb684
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasGraphUtilsV1.java 49d5a080158e3e54fd917a13f743f1cf495e66b0
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 09f69dbe6f519d84ae8e223532fc5adaa411ed9b
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasDeleteHandlerV1Test.java 555f0acc22c4bec67f1a2a4064acad66e939747a
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateHardDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateSoftDeleteV1Test.java PRE-CREATION
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/InverseReferenceUpdateV1Test.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/56860/diff/
>
>
> Testing
> -------
>
> Ran all unit and integration tests with no regressions.
>
>
> Thanks,
>
> David Kantor
>
>