You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Madhan Neethiraj <ma...@apache.org> on 2017/01/31 01:07:25 UTC
Review Request 56105: ATLAS-1503: updated AtlasObjectId to support
unqiueAttributes to identity the object
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56105/
-----------------------------------------------------------
Review request for atlas.
Bugs: ATLAS-1503
https://issues.apache.org/jira/browse/ATLAS-1503
Repository: atlas
Description
-------
Attribute uniqueAttribues is now added to AtlasObjectId.
Diffs
-----
intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java 4896e9d
intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java a25f3f6
intg/src/test/java/org/apache/atlas/type/TestAtlasObjectIdType.java 7d55233
Diff: https://reviews.apache.org/r/56105/diff/
Testing
-------
- updated unit tests to cover the changes to AtlasObjectId
Thanks,
Madhan Neethiraj
Re: Review Request 56105: ATLAS-1506: updated AtlasObjectId to support
unqiueAttributes to identity the object
Posted by Madhan Neethiraj <ma...@apache.org>.
> On Jan. 31, 2017, 10:22 a.m., David Radley wrote:
> > intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java, line 555
> > <https://reviews.apache.org/r/56105/diff/1/?file=1619640#file1619640line555>
> >
> > I am wondering what happens if you specify an attribute that has an entity as its type, if the target entity has an attribute pointing back to our entity - wont this loop. I think we need to restrict the attribute type in the objectid to numbers or strings. I like the idea of a list of attributes.
David - yes, allowing object_id attributes to have references to other objects can cause loop. It will be best to restrict these attributes to primitive types only.
- Madhan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56105/#review163642
-----------------------------------------------------------
On Jan. 31, 2017, 1:08 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56105/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 1:08 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1506
> https://issues.apache.org/jira/browse/ATLAS-1506
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Attribute uniqueAttribues is now added to AtlasObjectId.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java 4896e9d
> intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java a25f3f6
> intg/src/test/java/org/apache/atlas/type/TestAtlasObjectIdType.java 7d55233
>
> Diff: https://reviews.apache.org/r/56105/diff/
>
>
> Testing
> -------
>
> - updated unit tests to cover the changes to AtlasObjectId
>
>
> Thanks,
>
> Madhan Neethiraj
>
>
Re: Review Request 56105: ATLAS-1506: updated AtlasObjectId to support
unqiueAttributes to identity the object
Posted by David Radley <da...@uk.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56105/#review163642
-----------------------------------------------------------
intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java (line 53)
<https://reviews.apache.org/r/56105/#comment235122>
I suggest that it would be more flexible to be able to specify any attribute, not just the unique ones. For example if you have a name field on an entity, it may not be unique but it would be very useful.
intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java (line 553)
<https://reviews.apache.org/r/56105/#comment235126>
I am wondering what happens if you specify an attribute that has an entity as its type, if the target entity has an attribute pointing back to our entity - wont this loop. I think we need to restrict the attribute type in the objectid to numbers or strings. I like the idea of a list of attributes.
- David Radley
On Jan. 31, 2017, 1:08 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56105/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 1:08 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1506
> https://issues.apache.org/jira/browse/ATLAS-1506
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Attribute uniqueAttribues is now added to AtlasObjectId.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java 4896e9d
> intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java a25f3f6
> intg/src/test/java/org/apache/atlas/type/TestAtlasObjectIdType.java 7d55233
>
> Diff: https://reviews.apache.org/r/56105/diff/
>
>
> Testing
> -------
>
> - updated unit tests to cover the changes to AtlasObjectId
>
>
> Thanks,
>
> Madhan Neethiraj
>
>
Re: Review Request 56105: ATLAS-1506: updated AtlasObjectId to support
unqiueAttributes to identity the object
Posted by Suma Shivaprasad <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56105/#review163626
-----------------------------------------------------------
Ship it!
Ship It!
- Suma Shivaprasad
On Jan. 31, 2017, 1:08 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56105/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 1:08 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1506
> https://issues.apache.org/jira/browse/ATLAS-1506
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Attribute uniqueAttribues is now added to AtlasObjectId.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java 4896e9d
> intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java a25f3f6
> intg/src/test/java/org/apache/atlas/type/TestAtlasObjectIdType.java 7d55233
>
> Diff: https://reviews.apache.org/r/56105/diff/
>
>
> Testing
> -------
>
> - updated unit tests to cover the changes to AtlasObjectId
>
>
> Thanks,
>
> Madhan Neethiraj
>
>
Re: Review Request 56105: ATLAS-1506: updated AtlasObjectId to support
unqiueAttributes to identity the object
Posted by Madhan Neethiraj <ma...@apache.org>.
> On Feb. 15, 2017, 11:51 p.m., Madhan Neethiraj wrote:
> > intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java, line 53
> > <https://reviews.apache.org/r/56105/diff/1/?file=1619639#file1619639line53>
> >
> > AtlasObjectId should point to an unique object; hence the use of 'uniqueAttributes' here. If a non-unique attribute is allowed, it can potentially match muliple objects - making AtlasObjectId ambigious.
>
> David Radley wrote:
> Policing attributes in graphs to be unique is problematic; how do you police uniqueness without big graph scans. If we label attributes as unique but do not police it - this is misleading as the attriubutes may not be unique in reality.
> Too many unique attributes would lead to a proliferation of indexes and associated performance implications. RDBs have slick mechanisms for dealing with primary keys and foreign keys and the like - they can readily do this as they are set based. This is why I think we should not be encouraging the use of unique attributes. I think being able to specify any attribute would help; for example a name is useful but not unique.
Some attributes, like employee_id/dept_id can be used to identify a specific entity instance. Contrast this with other attributes, like comment/description/createTime, which can't be used identify a specific instance.
Wouldn't it be useful to distinguish attributes based on above criteria?
- Madhan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56105/#review165780
-----------------------------------------------------------
On Jan. 31, 2017, 1:08 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56105/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 1:08 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1506
> https://issues.apache.org/jira/browse/ATLAS-1506
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Attribute uniqueAttribues is now added to AtlasObjectId.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java 4896e9d
> intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java a25f3f6
> intg/src/test/java/org/apache/atlas/type/TestAtlasObjectIdType.java 7d55233
>
> Diff: https://reviews.apache.org/r/56105/diff/
>
>
> Testing
> -------
>
> - updated unit tests to cover the changes to AtlasObjectId
>
>
> Thanks,
>
> Madhan Neethiraj
>
>
Re: Review Request 56105: ATLAS-1506: updated AtlasObjectId to support
unqiueAttributes to identity the object
Posted by David Radley <da...@uk.ibm.com>.
> On Feb. 15, 2017, 11:51 p.m., Madhan Neethiraj wrote:
> > intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java, line 53
> > <https://reviews.apache.org/r/56105/diff/1/?file=1619639#file1619639line53>
> >
> > AtlasObjectId should point to an unique object; hence the use of 'uniqueAttributes' here. If a non-unique attribute is allowed, it can potentially match muliple objects - making AtlasObjectId ambigious.
Policing attributes in graphs to be unique is problematic; how do you police uniqueness without big graph scans. If we label attributes as unique but do not police it - this is misleading as the attriubutes may not be unique in reality.
Too many unique attributes would lead to a proliferation of indexes and associated performance implications. RDBs have slick mechanisms for dealing with primary keys and foreign keys and the like - they can readily do this as they are set based. This is why I think we should not be encouraging the use of unique attributes. I think being able to specify any attribute would help; for example a name is useful but not unique.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56105/#review165780
-----------------------------------------------------------
On Jan. 31, 2017, 1:08 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56105/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 1:08 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1506
> https://issues.apache.org/jira/browse/ATLAS-1506
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Attribute uniqueAttribues is now added to AtlasObjectId.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java 4896e9d
> intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java a25f3f6
> intg/src/test/java/org/apache/atlas/type/TestAtlasObjectIdType.java 7d55233
>
> Diff: https://reviews.apache.org/r/56105/diff/
>
>
> Testing
> -------
>
> - updated unit tests to cover the changes to AtlasObjectId
>
>
> Thanks,
>
> Madhan Neethiraj
>
>
Re: Review Request 56105: ATLAS-1506: updated AtlasObjectId to support
unqiueAttributes to identity the object
Posted by David Radley <da...@uk.ibm.com>.
> On Feb. 15, 2017, 11:51 p.m., Madhan Neethiraj wrote:
> > intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java, line 53
> > <https://reviews.apache.org/r/56105/diff/1/?file=1619639#file1619639line53>
> >
> > AtlasObjectId should point to an unique object; hence the use of 'uniqueAttributes' here. If a non-unique attribute is allowed, it can potentially match muliple objects - making AtlasObjectId ambigious.
>
> David Radley wrote:
> Policing attributes in graphs to be unique is problematic; how do you police uniqueness without big graph scans. If we label attributes as unique but do not police it - this is misleading as the attriubutes may not be unique in reality.
> Too many unique attributes would lead to a proliferation of indexes and associated performance implications. RDBs have slick mechanisms for dealing with primary keys and foreign keys and the like - they can readily do this as they are set based. This is why I think we should not be encouraging the use of unique attributes. I think being able to specify any attribute would help; for example a name is useful but not unique.
>
> Madhan Neethiraj wrote:
> Some attributes, like employee_id/dept_id can be used to identify a specific entity instance. Contrast this with other attributes, like comment/description/createTime, which can't be used identify a specific instance.
>
> Wouldn't it be useful to distinguish attributes based on above criteria?
If Atlas could trust the source system to police the field to be unique (so Atlas does not need to police it), for example if this was an RDB foreign key; this would be very useful.
For other cases when there is no unique attribute, then adding a non-unique attribute would also be useful
For glossary terms and categories (an area,I am looking into in ATLAS-1410), they have no unique attributes apart from the guid, they do have name attribute as a human readable way of identifying an object instance, but is probably not unique. I was hoping that this fix would allow me to include the name field in the relationship - this turned out to be very useful for debugging in my prototyping.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56105/#review165780
-----------------------------------------------------------
On Jan. 31, 2017, 1:08 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56105/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 1:08 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1506
> https://issues.apache.org/jira/browse/ATLAS-1506
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Attribute uniqueAttribues is now added to AtlasObjectId.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java 4896e9d
> intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java a25f3f6
> intg/src/test/java/org/apache/atlas/type/TestAtlasObjectIdType.java 7d55233
>
> Diff: https://reviews.apache.org/r/56105/diff/
>
>
> Testing
> -------
>
> - updated unit tests to cover the changes to AtlasObjectId
>
>
> Thanks,
>
> Madhan Neethiraj
>
>
Re: Review Request 56105: ATLAS-1506: updated AtlasObjectId to support
unqiueAttributes to identity the object
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56105/#review165780
-----------------------------------------------------------
intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java (line 53)
<https://reviews.apache.org/r/56105/#comment237635>
AtlasObjectId should point to an unique object; hence the use of 'uniqueAttributes' here. If a non-unique attribute is allowed, it can potentially match muliple objects - making AtlasObjectId ambigious.
intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java (line 553)
<https://reviews.apache.org/r/56105/#comment237636>
In general, allowing entity type attributes to have value of entire entity is trouble! ATLAS-1522 addresses this concern - by restricting such attributes to hold only objectid as value. Please review this JIRA for details.
With this update, attribute values in AtlasObjectId.uniqueAttributes will only have objectid value - and not an entire entity.
- Madhan Neethiraj
On Jan. 31, 2017, 1:08 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56105/
> -----------------------------------------------------------
>
> (Updated Jan. 31, 2017, 1:08 a.m.)
>
>
> Review request for atlas.
>
>
> Bugs: ATLAS-1506
> https://issues.apache.org/jira/browse/ATLAS-1506
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Attribute uniqueAttribues is now added to AtlasObjectId.
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java 4896e9d
> intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java a25f3f6
> intg/src/test/java/org/apache/atlas/type/TestAtlasObjectIdType.java 7d55233
>
> Diff: https://reviews.apache.org/r/56105/diff/
>
>
> Testing
> -------
>
> - updated unit tests to cover the changes to AtlasObjectId
>
>
> Thanks,
>
> Madhan Neethiraj
>
>
Re: Review Request 56105: ATLAS-1506: updated AtlasObjectId to support
unqiueAttributes to identity the object
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56105/
-----------------------------------------------------------
(Updated Jan. 31, 2017, 1:08 a.m.)
Review request for atlas.
Changes
-------
fixed incorrect JIRA #
Summary (updated)
-----------------
ATLAS-1506: updated AtlasObjectId to support unqiueAttributes to identity the object
Bugs: ATLAS-1506
https://issues.apache.org/jira/browse/ATLAS-1506
Repository: atlas
Description
-------
Attribute uniqueAttribues is now added to AtlasObjectId.
Diffs
-----
intg/src/main/java/org/apache/atlas/model/instance/AtlasObjectId.java 4896e9d
intg/src/main/java/org/apache/atlas/type/AtlasBuiltInTypes.java a25f3f6
intg/src/test/java/org/apache/atlas/type/TestAtlasObjectIdType.java 7d55233
Diff: https://reviews.apache.org/r/56105/diff/
Testing
-------
- updated unit tests to cover the changes to AtlasObjectId
Thanks,
Madhan Neethiraj