You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@atlas.apache.org by anshu shukla <an...@gmail.com> on 2018/08/10 14:13:13 UTC
Lineage for updation in Process or any vertex
Hi,
Is there any inbuilt feature in Atlas to track the changes in the process
or any other custom vertex. Say in some use case there is some entity with
some X1 description and now after some time we need to update the
description to X2. is there any default way to keep track of the version
and lineage. *I can see the audit trail in UI but no lineage graph for it. *
The above use case can be more complex if we include other types of
vertices and their lineage and impact on others. This lineage is different
from the process and dataset lineage already exposed by the framework.
*Is there any version number associated with every vertex that
increments on every update of the same GUID?*
--
Thanks & Regards,
Anshu Shukla
Re: Lineage for updation in Process or any vertex
Posted by Madhan Neethiraj <ma...@apache.org>.
Anshu,
After reviewing the code, I find that ‘version’ field is not updated during entity-update operation. This is a bug. The correct behavior should be – whenever an entity is updated, Atlas should automatically increment the entity version.
Can you please file a JIRA to track this fix?
About accessing Atlas data in JanusGraph directly – I would strongly recommend against this, as these details are Atlas internal that can change without considering applications that depend on the graph level details. The recommended way is to access data via Atlas APIs, which will ensure that user have access to only authorized data (based on authorization policies).
Regards,
Madhan
From: anshu shukla <an...@gmail.com>
Reply-To: "user@atlas.apache.org" <us...@atlas.apache.org>
Date: Saturday, August 11, 2018 at 9:40 PM
To: "user@atlas.apache.org" <us...@atlas.apache.org>
Subject: Re: Lineage for updation in Process or any vertex
Also here it says the something else about the version number.
"In short, it is used for maintenance of types. Beyond this, version is NOT used internally for validation, etc. "
Link- https://community.hortonworks.com/questions/163756/how-does-apache-atlas-version-entity-metadata.html
On Sun, Aug 12, 2018 at 9:52 AM anshu shukla <an...@gmail.com> wrote:
Hi, Thanks for replying!
As mentioned by you, I tried to use setVersion() for an entity but it's not reflecting on to the DB. Every time when I read it again its set to 0 again.
Q1- How can I directly query the Janus graph to see what is persisted in it. Also, I am unable to see how call graph trace to graph API to see how updates to entities are reflected in the backend.
Q2- DO we need to do anything else also after say dbEntity.setVersion(new Random().nextLong()); function call to persisit the updated the version number permamanently.
On Sat, Aug 11, 2018 at 2:17 AM Madhan Neethiraj <ma...@apache.org> wrote:
Anshu,
There is no visualization of the changes to an entity attributes across multiple versions; though such visualization can be developed from the entity states stored in audit logs.
There is a version number associated with each entity; but currently it is not incremented automatically during entity update. It is left to the API caller to set appropriate version number. I think it could be useful to increment entity version automatically (similar to how AtlasEntity.updateTime is updated). If this is critical for your usecase, can you please file a JIRA?
Hope this helps.
Madhan
From: anshu shukla <an...@gmail.com>
Reply-To: "user@atlas.apache.org" <us...@atlas.apache.org>
Date: Friday, August 10, 2018 at 7:13 AM
To: "user@atlas.apache.org" <us...@atlas.apache.org>
Subject: Lineage for updation in Process or any vertex
Hi,
Is there any inbuilt feature in Atlas to track the changes in the process or any other custom vertex. Say in some use case there is some entity with some X1 description and now after some time we need to update the description to X2. is there any default way to keep track of the version and lineage. I can see the audit trail in UI but no lineage graph for it.
The above use case can be more complex if we include other types of vertices and their lineage and impact on others. This lineage is different from the process and dataset lineage already exposed by the framework.
Is there any version number associated with every vertex that increments on every update of the same GUID?
--
Thanks & Regards,
Anshu Shukla
--
Thanks & Regards,
Anshu Shukla
--
Thanks & Regards,
Anshu Shukla
Re: Lineage for updation in Process or any vertex
Posted by anshu shukla <an...@gmail.com>.
Also here it says the something else about the version number.
* "In short, it is used for maintenance of types. Beyond this, version is
NOT used internally for validation, etc. "*
Link-
https://community.hortonworks.com/questions/163756/how-does-apache-atlas-version-entity-metadata.html
On Sun, Aug 12, 2018 at 9:52 AM anshu shukla <an...@gmail.com> wrote:
> Hi, Thanks for replying!
>
> As mentioned by you, I tried to use setVersion() for an entity but it's
> not reflecting on to the DB. Every time when I read it again its set to 0
> again.
>
> *Q1-* How can I directly query the Janus graph to see what is persisted
> in it. Also, I am unable to see how call graph trace to graph API to see
> how updates to entities are reflected in the backend.
> *Q2-* DO we need to do anything else also after say *dbEntity.setVersion(new
> Random().nextLong());* function call to persisit the updated the version
> number permamanently.
>
>
> On Sat, Aug 11, 2018 at 2:17 AM Madhan Neethiraj <ma...@apache.org>
> wrote:
>
>> Anshu,
>>
>>
>>
>> There is no visualization of the changes to an entity attributes across
>> multiple versions; though such visualization can be developed from the
>> entity states stored in audit logs.
>>
>>
>>
>> There is a version number associated with each entity; but currently it
>> is not incremented automatically during entity update. It is left to the
>> API caller to set appropriate version number. I think it could be useful to
>> increment entity version automatically (similar to how
>> AtlasEntity.updateTime is updated). If this is critical for your usecase,
>> can you please file a JIRA?
>>
>>
>>
>> Hope this helps.
>>
>>
>>
>> Madhan
>>
>>
>>
>>
>>
>>
>>
>> *From: *anshu shukla <an...@gmail.com>
>> *Reply-To: *"user@atlas.apache.org" <us...@atlas.apache.org>
>> *Date: *Friday, August 10, 2018 at 7:13 AM
>> *To: *"user@atlas.apache.org" <us...@atlas.apache.org>
>> *Subject: *Lineage for updation in Process or any vertex
>>
>>
>>
>> Hi,
>>
>>
>>
>> Is there any inbuilt feature in Atlas to track the changes in the process
>> or any other custom vertex. Say in some use case there is some entity with
>> some X1 description and now after some time we need to update the
>> description to X2. is there any default way to keep track of the version
>> and lineage. *I can see the audit trail in UI but no lineage graph for
>> it. *
>>
>>
>>
>> The above use case can be more complex if we include other types of
>> vertices and their lineage and impact on others. This lineage is different
>> from the process and dataset lineage already exposed by the framework.
>>
>>
>>
>> *Is there any version number associated with every vertex that
>> increments on every update of the same GUID?*
>>
>>
>>
>> --
>>
>> Thanks & Regards,
>> Anshu Shukla
>>
>
>
> --
> Thanks & Regards,
> Anshu Shukla
>
--
Thanks & Regards,
Anshu Shukla
Re: Lineage for updation in Process or any vertex
Posted by anshu shukla <an...@gmail.com>.
Hi, Thanks for replying!
As mentioned by you, I tried to use setVersion() for an entity but it's not
reflecting on to the DB. Every time when I read it again its set to 0 again.
*Q1-* How can I directly query the Janus graph to see what is persisted in
it. Also, I am unable to see how call graph trace to graph API to see how
updates to entities are reflected in the backend.
*Q2-* DO we need to do anything else also after say *dbEntity.setVersion(new
Random().nextLong());* function call to persisit the updated the version
number permamanently.
On Sat, Aug 11, 2018 at 2:17 AM Madhan Neethiraj <ma...@apache.org> wrote:
> Anshu,
>
>
>
> There is no visualization of the changes to an entity attributes across
> multiple versions; though such visualization can be developed from the
> entity states stored in audit logs.
>
>
>
> There is a version number associated with each entity; but currently it is
> not incremented automatically during entity update. It is left to the API
> caller to set appropriate version number. I think it could be useful to
> increment entity version automatically (similar to how
> AtlasEntity.updateTime is updated). If this is critical for your usecase,
> can you please file a JIRA?
>
>
>
> Hope this helps.
>
>
>
> Madhan
>
>
>
>
>
>
>
> *From: *anshu shukla <an...@gmail.com>
> *Reply-To: *"user@atlas.apache.org" <us...@atlas.apache.org>
> *Date: *Friday, August 10, 2018 at 7:13 AM
> *To: *"user@atlas.apache.org" <us...@atlas.apache.org>
> *Subject: *Lineage for updation in Process or any vertex
>
>
>
> Hi,
>
>
>
> Is there any inbuilt feature in Atlas to track the changes in the process
> or any other custom vertex. Say in some use case there is some entity with
> some X1 description and now after some time we need to update the
> description to X2. is there any default way to keep track of the version
> and lineage. *I can see the audit trail in UI but no lineage graph for
> it. *
>
>
>
> The above use case can be more complex if we include other types of
> vertices and their lineage and impact on others. This lineage is different
> from the process and dataset lineage already exposed by the framework.
>
>
>
> *Is there any version number associated with every vertex that
> increments on every update of the same GUID?*
>
>
>
> --
>
> Thanks & Regards,
> Anshu Shukla
>
--
Thanks & Regards,
Anshu Shukla
Re: Lineage for updation in Process or any vertex
Posted by Madhan Neethiraj <ma...@apache.org>.
Anshu,
There is no visualization of the changes to an entity attributes across multiple versions; though such visualization can be developed from the entity states stored in audit logs.
There is a version number associated with each entity; but currently it is not incremented automatically during entity update. It is left to the API caller to set appropriate version number. I think it could be useful to increment entity version automatically (similar to how AtlasEntity.updateTime is updated). If this is critical for your usecase, can you please file a JIRA?
Hope this helps.
Madhan
From: anshu shukla <an...@gmail.com>
Reply-To: "user@atlas.apache.org" <us...@atlas.apache.org>
Date: Friday, August 10, 2018 at 7:13 AM
To: "user@atlas.apache.org" <us...@atlas.apache.org>
Subject: Lineage for updation in Process or any vertex
Hi,
Is there any inbuilt feature in Atlas to track the changes in the process or any other custom vertex. Say in some use case there is some entity with some X1 description and now after some time we need to update the description to X2. is there any default way to keep track of the version and lineage. I can see the audit trail in UI but no lineage graph for it.
The above use case can be more complex if we include other types of vertices and their lineage and impact on others. This lineage is different from the process and dataset lineage already exposed by the framework.
Is there any version number associated with every vertex that increments on every update of the same GUID?
--
Thanks & Regards,
Anshu Shukla