You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Shwetha G S (JIRA)" <ji...@apache.org> on 2016/04/28 08:06:12 UTC
[jira] [Assigned] (ATLAS-710) Soft deletes: GET response on a
deleted hive tables should have references to hive_db and hive_storagedesc
[ https://issues.apache.org/jira/browse/ATLAS-710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shwetha G S reassigned ATLAS-710:
---------------------------------
Assignee: Shwetha G S
> Soft deletes: GET response on a deleted hive tables should have references to hive_db and hive_storagedesc
> ----------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-710
> URL: https://issues.apache.org/jira/browse/ATLAS-710
> Project: Atlas
> Issue Type: Bug
> Affects Versions: 0.7-incubating
> Reporter: Ayub Khan
> Assignee: Shwetha G S
>
> Soft deletes: GET response on a deleted hive tables does not have references to hive_db and hive_storagedesc.
> After creating the hive_table, get response shows references to db, sd and columns
> {noformat}
> curl 'http://localhost:21000/api/atlas/entities/1b11b234-7360-4679-93f9-d1c01f839f79' -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36' -H 'Accept: application/json, text/plain, */*' -H 'Referer: http://localhost:21000/index.html' -H 'Cookie: JSESSIONID=3frbae3qwzi2xkdjyvz8nyko' -H 'Connection: keep-alive' --compressed | python -m json.tool
> % Total % Received % Xferd Average Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 100 3271 0 3271 0 0 2088 0 --:--:-- 0:00:01 --:--:-- 2088
> {
> "GUID": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "definition": {
> "id": {
> "id": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "2c50426e-7db3-4912-9bbc-13150e962ed5",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "name",
> "qualifiedName": "default.after_changes_13.name@primary",
> "type": "string"
> }
> },
> {
> "id": {
> "id": "2af81d25-bbcd-44bd-95b8-4fee915c9dba",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "age",
> "qualifiedName": "default.after_changes_13.age@primary",
> "type": "int"
> }
> }
> ],
> "comment": null,
> "createTime": "2016-04-26T09:27:29.000Z",
> "db": {
> "id": "b23cc76a-32e4-4137-860e-8cac2689ba0e",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_db",
> "version": 0
> },
> "description": null,
> "lastAccessTime": "2016-04-26T09:27:29.000Z",
> "name": "default.after_changes_13@primary",
> "owner": "apathan",
> "parameters": {
> "transient_lastDdlTime": "1461662849"
> },
> "partitionKeys": [
> {
> "id": {
> "id": "84db2789-82b3-460e-a827-ee8f94c786cc",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "ds",
> "qualifiedName": "default.after_changes_13.ds@primary",
> "type": "string"
> }
> }
> ],
> "retention": 0,
> "sd": {
> "id": {
> "id": "5ebc8823-ecab-4c2d-8f0d-17a861984070",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_storagedesc",
> "version": 0
> },
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_storagedesc",
> "values": {
> "bucketCols": null,
> "compressed": false,
> "inputFormat": "org.apache.hadoop.mapred.TextInputFormat",
> "location": "hdfs://localhost:9000/user/hive/warehouse/after_changes_13",
> "numBuckets": -1,
> "outputFormat": "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat",
> "parameters": null,
> "qualifiedName": "default.after_changes_13@primary_storage",
> "serdeInfo": {
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "hive_serde",
> "values": {
> "name": null,
> "parameters": {
> "serialization.format": "1"
> },
> "serializationLib": "org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"
> }
> },
> "sortCols": null,
> "storedAsSubDirectories": false
> }
> },
> "tableName": "after_changes_13",
> "tableType": "MANAGED_TABLE",
> "temporary": false,
> "viewExpandedText": null,
> "viewOriginalText": null
> }
> },
> "requestId": "qtp835142742-2359 - ccef18a2-916c-48b1-b629-a330c5689a69"
> }
> {noformat}
> After deleting the table, db and storage descriptor references are missing from the table entity response.
> {noformat}
> apathan@Ayub-Pathans-Macbook-Pro:~ > curl 'http://localhost:21000/api/atlas/entities/1b11b234-7360-4679-93f9-d1c01f839f79' -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36' -H 'Accept: application/json, text/plain, */*' -H 'Referer: http://localhost:21000/index.html' -H 'Cookie: JSESSIONID=3frbae3qwzi2xkdjyvz8nyko' -H 'Connection: keep-alive' --compressed | python -m json.tool
> % Total % Received % Xferd Average Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 100 2142 0 2142 0 0 51088 0 --:--:-- --:--:-- --:--:-- 52243
> {
> "GUID": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "definition": {
> "id": {
> "id": "1b11b234-7360-4679-93f9-d1c01f839f79",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "DELETED",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "2c50426e-7db3-4912-9bbc-13150e962ed5",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "DELETED",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "name",
> "qualifiedName": "default.after_changes_13.name@primary",
> "type": "string"
> }
> },
> {
> "id": {
> "id": "2af81d25-bbcd-44bd-95b8-4fee915c9dba",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "DELETED",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "age",
> "qualifiedName": "default.after_changes_13.age@primary",
> "type": "int"
> }
> }
> ],
> "comment": null,
> "createTime": "2016-04-26T09:27:29.000Z",
> "db": null,
> "description": null,
> "lastAccessTime": "2016-04-26T09:27:29.000Z",
> "name": "default.after_changes_13@primary",
> "owner": "apathan",
> "parameters": {
> "transient_lastDdlTime": "1461662849"
> },
> "partitionKeys": [
> {
> "id": {
> "id": "84db2789-82b3-460e-a827-ee8f94c786cc",
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "DELETED",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "ds",
> "qualifiedName": "default.after_changes_13.ds@primary",
> "type": "string"
> }
> }
> ],
> "retention": 0,
> "sd": null,
> "tableName": "after_changes_13",
> "tableType": "MANAGED_TABLE",
> "temporary": false,
> "viewExpandedText": null,
> "viewOriginalText": null
> }
> },
> "requestId": "qtp835142742-2432 - efe83211-ba90-4193-a644-a9d50dc9d410"
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)