You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "chaitali borole (Jira)" <ji...@apache.org> on 2024/08/25 08:41:00 UTC
[jira] [Updated] (ATLAS-4802) Atlas 'updateTime' parameter is not updated when business metadata, labels is edited.
[ https://issues.apache.org/jira/browse/ATLAS-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
chaitali borole updated ATLAS-4802:
-----------------------------------
Description:
When updating the business metadata on an atlas entity, the 'updateTime' system attribute is not changed.
Updates to other aspects, such as technical properties and user defined attributes does update the timestamp, but not the business metadata.
This appears to be a issue with the core Atlas product.
curl -u admin:hadoop12345! -X GET --header 'Content-Type: application/json;charset=UTF-8' "http://localhost:21000/api/atlas/v2/entity/guid/5b4e426e-404f-4a10-abab-f0b3879c0acb"
{"referredEntities":{},"entity":{"typeName":"hive_column","attributes":{"owner":"hive","replicatedTo":null,"userDescription":null,"replicatedFrom":null,"qualifiedName":"sys.mv_creation_metadata.txn_list@cm","displayName":null,"name":"txn_list","description":null,"comment":"from deserializer","position":3,"type":"string","table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table"}},"guid":"5b4e426e-404f-4a10-abab-f0b3879c0acb","isIncomplete":false,"status":"ACTIVE","createdBy":"hive","updatedBy":"hive","createTime":1697203083822,"updateTime":1697203083822,"version":0,"relationshipAttributes":{"inputToProcesses":[],"pipeline":null,"schema":[],"model":null,"meanings":[],"table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table","entityStatus":"ACTIVE","displayText":"mv_creation_metadata","relationshipType":"hive_table_columns","relationshipGuid":"39f22e08-8ff2-4bd3-9a2f-9e2efce69fba","relationshipStatus":"ACTIVE","relationshipAttributes":{"typeName":"hive_table_columns"}},"outputFromProcesses":[]},"labels":[]}}
--------
After adding business metadata "text : value1" to the entity txn_list (hive_column),
--------
curl -u admin:hadoop12345! -X GET --header 'Content-Type: application/json;charset=UTF-8' "http://localhost:21000/api/atlas/v2/entity/guid/5b4e426e-404f-4a10-abab-f0b3879c0acb"
{"referredEntities":{},"entity":{"typeName":"hive_column","attributes":{"owner":"hive","replicatedTo":null,"userDescription":null,"replicatedFrom":null,"qualifiedName":"sys.mv_creation_metadata.txn_list@cm","displayName":null,"name":"txn_list","description":null,"comment":"from deserializer","position":3,"type":"string","table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table"}},"guid":"5b4e426e-404f-4a10-abab-f0b3879c0acb","isIncomplete":false,"status":"ACTIVE","createdBy":"hive","updatedBy":"hive","createTime":1697203083822,"updateTime":1697203083822,"version":0,"relationshipAttributes":{"inputToProcesses":[],"pipeline":null,"schema":[],"model":null,"meanings":[],"table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table","entityStatus":"ACTIVE","displayText":"mv_creation_metadata","relationshipType":"hive_table_columns","relationshipGuid":"39f22e08-8ff2-4bd3-9a2f-9e2efce69fba","relationshipStatus":"ACTIVE","relationshipAttributes":{"typeName":"hive_table_columns"}},"outputFromProcesses":[]},"businessAttributes":{"demo":{"text":"value1"}},"labels":[]}}
--------
As seen "updateTime":1697203083822 still remains the same.
However, if we add an user-defined property "age : 20",
--------
curl -u admin:hadoop12345! -X GET --header 'Content-Type: application/json;charset=UTF-8' "http://localhost:21000/api/atlas/v2/entity/guid/5b4e426e-404f-4a10-abab-f0b3879c0acb"
{"referredEntities":{},"entity":{"typeName":"hive_column","attributes":{"owner":"hive","replicatedTo":null,"userDescription":null,"replicatedFrom":null,"qualifiedName":"sys.mv_creation_metadata.txn_list@cm","displayName":null,"name":"txn_list","description":null,"comment":"from deserializer","position":3,"type":"string","table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table"}},"guid":"5b4e426e-404f-4a10-abab-f0b3879c0acb","isIncomplete":false,"status":"ACTIVE","createdBy":"hive","updatedBy":"admin","createTime":1697203083822,"updateTime":1697203987465,"version":0,"relationshipAttributes":{"inputToProcesses":[],"pipeline":null,"schema":[],"model":null,"meanings":[],"table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table","entityStatus":"ACTIVE","displayText":"mv_creation_metadata","relationshipType":"hive_table_columns","relationshipGuid":"39f22e08-8ff2-4bd3-9a2f-9e2efce69fba","relationshipStatus":"ACTIVE","relationshipAttributes":{"typeName":"hive_table_columns"}},"outputFromProcesses":[]},"customAttributes":{"age":"20"},"businessAttributes":{"demo":{"text":"value1"}},"labels":[]}}
Scenario 2:
When label / business metadata is added to/deleted from entity, lastupdatetime is not updated , due to which incremental export will not honor the update and create empty zip.
was:
When updating the business metadata on an atlas entity, the 'updateTime' system attribute is not changed.
Updates to other aspects, such as technical properties and user defined attributes does update the timestamp, but not the business metadata.
This appears to be a issue with the core Atlas product.
curl -u admin:hadoop12345! -X GET --header 'Content-Type: application/json;charset=UTF-8' "http://localhost:21000/api/atlas/v2/entity/guid/5b4e426e-404f-4a10-abab-f0b3879c0acb"
{"referredEntities":{},"entity":{"typeName":"hive_column","attributes":{"owner":"hive","replicatedTo":null,"userDescription":null,"replicatedFrom":null,"qualifiedName":"sys.mv_creation_metadata.txn_list@cm","displayName":null,"name":"txn_list","description":null,"comment":"from deserializer","position":3,"type":"string","table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table"}},"guid":"5b4e426e-404f-4a10-abab-f0b3879c0acb","isIncomplete":false,"status":"ACTIVE","createdBy":"hive","updatedBy":"hive","createTime":1697203083822,"updateTime":1697203083822,"version":0,"relationshipAttributes":{"inputToProcesses":[],"pipeline":null,"schema":[],"model":null,"meanings":[],"table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table","entityStatus":"ACTIVE","displayText":"mv_creation_metadata","relationshipType":"hive_table_columns","relationshipGuid":"39f22e08-8ff2-4bd3-9a2f-9e2efce69fba","relationshipStatus":"ACTIVE","relationshipAttributes":{"typeName":"hive_table_columns"}},"outputFromProcesses":[]},"labels":[]}}
--------
After adding business metadata "text : value1" to the entity txn_list (hive_column),
--------
curl -u admin:hadoop12345! -X GET --header 'Content-Type: application/json;charset=UTF-8' "http://localhost:21000/api/atlas/v2/entity/guid/5b4e426e-404f-4a10-abab-f0b3879c0acb"
{"referredEntities":{},"entity":{"typeName":"hive_column","attributes":{"owner":"hive","replicatedTo":null,"userDescription":null,"replicatedFrom":null,"qualifiedName":"sys.mv_creation_metadata.txn_list@cm","displayName":null,"name":"txn_list","description":null,"comment":"from deserializer","position":3,"type":"string","table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table"}},"guid":"5b4e426e-404f-4a10-abab-f0b3879c0acb","isIncomplete":false,"status":"ACTIVE","createdBy":"hive","updatedBy":"hive","createTime":1697203083822,"updateTime":1697203083822,"version":0,"relationshipAttributes":{"inputToProcesses":[],"pipeline":null,"schema":[],"model":null,"meanings":[],"table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table","entityStatus":"ACTIVE","displayText":"mv_creation_metadata","relationshipType":"hive_table_columns","relationshipGuid":"39f22e08-8ff2-4bd3-9a2f-9e2efce69fba","relationshipStatus":"ACTIVE","relationshipAttributes":{"typeName":"hive_table_columns"}},"outputFromProcesses":[]},"businessAttributes":{"demo":{"text":"value1"}},"labels":[]}}
--------
As seen "updateTime":1697203083822 still remains the same.
However, if we add an user-defined property "age : 20",
--------
curl -u admin:hadoop12345! -X GET --header 'Content-Type: application/json;charset=UTF-8' "http://localhost:21000/api/atlas/v2/entity/guid/5b4e426e-404f-4a10-abab-f0b3879c0acb"
{"referredEntities":{},"entity":{"typeName":"hive_column","attributes":{"owner":"hive","replicatedTo":null,"userDescription":null,"replicatedFrom":null,"qualifiedName":"sys.mv_creation_metadata.txn_list@cm","displayName":null,"name":"txn_list","description":null,"comment":"from deserializer","position":3,"type":"string","table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table"}},"guid":"5b4e426e-404f-4a10-abab-f0b3879c0acb","isIncomplete":false,"status":"ACTIVE","createdBy":"hive","updatedBy":"admin","createTime":1697203083822,"updateTime":1697203987465,"version":0,"relationshipAttributes":{"inputToProcesses":[],"pipeline":null,"schema":[],"model":null,"meanings":[],"table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table","entityStatus":"ACTIVE","displayText":"mv_creation_metadata","relationshipType":"hive_table_columns","relationshipGuid":"39f22e08-8ff2-4bd3-9a2f-9e2efce69fba","relationshipStatus":"ACTIVE","relationshipAttributes":{"typeName":"hive_table_columns"}},"outputFromProcesses":[]},"customAttributes":{"age":"20"},"businessAttributes":{"demo":{"text":"value1"}},"labels":[]}}
> Atlas 'updateTime' parameter is not updated when business metadata, labels is edited.
> -------------------------------------------------------------------------------------
>
> Key: ATLAS-4802
> URL: https://issues.apache.org/jira/browse/ATLAS-4802
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: 3.0.0
> Reporter: chaitali borole
> Assignee: chaitali borole
> Priority: Major
>
> When updating the business metadata on an atlas entity, the 'updateTime' system attribute is not changed.
> Updates to other aspects, such as technical properties and user defined attributes does update the timestamp, but not the business metadata.
> This appears to be a issue with the core Atlas product.
> curl -u admin:hadoop12345! -X GET --header 'Content-Type: application/json;charset=UTF-8' "http://localhost:21000/api/atlas/v2/entity/guid/5b4e426e-404f-4a10-abab-f0b3879c0acb"
> {"referredEntities":{},"entity":{"typeName":"hive_column","attributes":{"owner":"hive","replicatedTo":null,"userDescription":null,"replicatedFrom":null,"qualifiedName":"sys.mv_creation_metadata.txn_list@cm","displayName":null,"name":"txn_list","description":null,"comment":"from deserializer","position":3,"type":"string","table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table"}},"guid":"5b4e426e-404f-4a10-abab-f0b3879c0acb","isIncomplete":false,"status":"ACTIVE","createdBy":"hive","updatedBy":"hive","createTime":1697203083822,"updateTime":1697203083822,"version":0,"relationshipAttributes":{"inputToProcesses":[],"pipeline":null,"schema":[],"model":null,"meanings":[],"table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table","entityStatus":"ACTIVE","displayText":"mv_creation_metadata","relationshipType":"hive_table_columns","relationshipGuid":"39f22e08-8ff2-4bd3-9a2f-9e2efce69fba","relationshipStatus":"ACTIVE","relationshipAttributes":{"typeName":"hive_table_columns"}},"outputFromProcesses":[]},"labels":[]}}
> --------
> After adding business metadata "text : value1" to the entity txn_list (hive_column),
> --------
> curl -u admin:hadoop12345! -X GET --header 'Content-Type: application/json;charset=UTF-8' "http://localhost:21000/api/atlas/v2/entity/guid/5b4e426e-404f-4a10-abab-f0b3879c0acb"
> {"referredEntities":{},"entity":{"typeName":"hive_column","attributes":{"owner":"hive","replicatedTo":null,"userDescription":null,"replicatedFrom":null,"qualifiedName":"sys.mv_creation_metadata.txn_list@cm","displayName":null,"name":"txn_list","description":null,"comment":"from deserializer","position":3,"type":"string","table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table"}},"guid":"5b4e426e-404f-4a10-abab-f0b3879c0acb","isIncomplete":false,"status":"ACTIVE","createdBy":"hive","updatedBy":"hive","createTime":1697203083822,"updateTime":1697203083822,"version":0,"relationshipAttributes":{"inputToProcesses":[],"pipeline":null,"schema":[],"model":null,"meanings":[],"table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table","entityStatus":"ACTIVE","displayText":"mv_creation_metadata","relationshipType":"hive_table_columns","relationshipGuid":"39f22e08-8ff2-4bd3-9a2f-9e2efce69fba","relationshipStatus":"ACTIVE","relationshipAttributes":{"typeName":"hive_table_columns"}},"outputFromProcesses":[]},"businessAttributes":{"demo":{"text":"value1"}},"labels":[]}}
> --------
> As seen "updateTime":1697203083822 still remains the same.
> However, if we add an user-defined property "age : 20",
> --------
> curl -u admin:hadoop12345! -X GET --header 'Content-Type: application/json;charset=UTF-8' "http://localhost:21000/api/atlas/v2/entity/guid/5b4e426e-404f-4a10-abab-f0b3879c0acb"
> {"referredEntities":{},"entity":{"typeName":"hive_column","attributes":{"owner":"hive","replicatedTo":null,"userDescription":null,"replicatedFrom":null,"qualifiedName":"sys.mv_creation_metadata.txn_list@cm","displayName":null,"name":"txn_list","description":null,"comment":"from deserializer","position":3,"type":"string","table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table"}},"guid":"5b4e426e-404f-4a10-abab-f0b3879c0acb","isIncomplete":false,"status":"ACTIVE","createdBy":"hive","updatedBy":"admin","createTime":1697203083822,"updateTime":1697203987465,"version":0,"relationshipAttributes":{"inputToProcesses":[],"pipeline":null,"schema":[],"model":null,"meanings":[],"table":{"guid":"bb0b34ae-4de7-4d9b-b072-e1b3c47fe14a","typeName":"hive_table","entityStatus":"ACTIVE","displayText":"mv_creation_metadata","relationshipType":"hive_table_columns","relationshipGuid":"39f22e08-8ff2-4bd3-9a2f-9e2efce69fba","relationshipStatus":"ACTIVE","relationshipAttributes":{"typeName":"hive_table_columns"}},"outputFromProcesses":[]},"customAttributes":{"age":"20"},"businessAttributes":{"demo":{"text":"value1"}},"labels":[]}}
> Scenario 2:
> When label / business metadata is added to/deleted from entity, lastupdatetime is not updated , due to which incremental export will not honor the update and create empty zip.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)