You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Madhan Neethiraj (JIRA)" <ji...@apache.org> on 2017/02/21 22:30:44 UTC
[jira] [Commented] (ATLAS-1576) Regression: Hbase table entity
creation fails with NPE at
org.apache.atlas.repository.converters.AtlasStructFormatConverter.fromV1ToV2
[ https://issues.apache.org/jira/browse/ATLAS-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876877#comment-15876877 ]
Madhan Neethiraj commented on ATLAS-1576:
-----------------------------------------
[~ayubkhan] - the failure is caused by presence of following unknown attributes in the message:
{noformat}
hbase_column_family.isUnique
hbase_column_family.uri
hbase_column.isUnique
hbase_column.uri
{noformat}
I will update Atlas to ignore such attributes, instead of failing to process the notification.
> Regression: Hbase table entity creation fails with NPE at org.apache.atlas.repository.converters.AtlasStructFormatConverter.fromV1ToV2
> --------------------------------------------------------------------------------------------------------------------------------------
>
> Key: ATLAS-1576
> URL: https://issues.apache.org/jira/browse/ATLAS-1576
> Project: Atlas
> Issue Type: Bug
> Components: atlas-intg
> Affects Versions: trunk, 0.8-incubating
> Reporter: Ayub Khan
> Assignee: Madhan Neethiraj
> Priority: Blocker
> Fix For: trunk, 0.8-incubating
>
>
> Hbase table entity is created using below command
> {noformat}
> echo '{"message": {"entities": [{"traitNames": [], "typeName": "hbase_table", "traits": {}, "values": {"owner": "hrt_qa", "qualifiedName": "hbase_table_aQTDg@cl1", "name": "hbase_table_aQTDg", "uri": "uris"}, "id": {"typeName": "hbase_table", "state": "ACTIVE", "version": 0, "id": "-74365953834994", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, {"traitNames": [], "typeName": "hbase_column_family", "traits": {}, "values": {"qualifiedName": "hbase_table_aQTDg.persData@cl1", "name": "persData", "owner": "hrt_qa", "table": {"traitNames": [], "typeName": "hbase_table", "traits": {}, "values": {"owner": "hrt_qa", "qualifiedName": "hbase_table_aQTDg@cl1", "name": "hbase_table_aQTDg", "uri": "uris"}, "id": {"typeName": "hbase_table", "state": "ACTIVE", "version": 0, "id": "-74365953834994", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column_family", "state": "ACTIVE", "version": 0, "id": "-009310583874", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, {"traitNames": [], "typeName": "hbase_column", "traits": {}, "values": {"column_family": {"traitNames": [], "typeName": "hbase_column_family", "traits": {}, "values": {"qualifiedName": "hbase_table_aQTDg.persData@cl1", "name": "persData", "owner": "hrt_qa", "table": {"traitNames": [], "typeName": "hbase_table", "traits": {}, "values": {"owner": "hrt_qa", "qualifiedName": "hbase_table_aQTDg@cl1", "name": "hbase_table_aQTDg", "uri": "uris"}, "id": {"typeName": "hbase_table", "state": "ACTIVE", "version": 0, "id": "-74365953834994", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column_family", "state": "ACTIVE", "version": 0, "id": "-009310583874", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "qualifiedName": "hbase_table_aQTDg.persData.name@cl1", "name": "name", "owner": "hrt_qa", "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column", "state": "ACTIVE", "version": 0, "id": "-507678968802", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, {"traitNames": [], "typeName": "hbase_column", "traits": {}, "values": {"column_family": {"traitNames": [], "typeName": "hbase_column_family", "traits": {}, "values": {"qualifiedName": "hbase_table_aQTDg.persData@cl1", "name": "persData", "owner": "hrt_qa", "table": {"traitNames": [], "typeName": "hbase_table", "traits": {}, "values": {"owner": "hrt_qa", "qualifiedName": "hbase_table_aQTDg@cl1", "name": "hbase_table_aQTDg", "uri": "uris"}, "id": {"typeName": "hbase_table", "state": "ACTIVE", "version": 0, "id": "-74365953834994", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column_family", "state": "ACTIVE", "version": 0, "id": "-009310583874", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "qualifiedName": "hbase_table_aQTDg.persData.age@cl1", "name": "age", "owner": "hrt_qa", "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column", "state": "ACTIVE", "version": 0, "id": "-400284865252", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, {"traitNames": [], "typeName": "hbase_column_family", "traits": {}, "values": {"qualifiedName": "hbase_table_aQTDg.profData@cl1", "name": "profData", "owner": "hrt_qa", "table": {"traitNames": [], "typeName": "hbase_table", "traits": {}, "values": {"owner": "hrt_qa", "qualifiedName": "hbase_table_aQTDg@cl1", "name": "hbase_table_aQTDg", "uri": "uris"}, "id": {"typeName": "hbase_table", "state": "ACTIVE", "version": 0, "id": "-74365953834994", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column_family", "state": "ACTIVE", "version": 0, "id": "-795512686603", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, {"traitNames": [], "typeName": "hbase_column", "traits": {}, "values": {"column_family": {"traitNames": [], "typeName": "hbase_column_family", "traits": {}, "values": {"qualifiedName": "hbase_table_aQTDg.profData@cl1", "name": "profData", "owner": "hrt_qa", "table": {"traitNames": [], "typeName": "hbase_table", "traits": {}, "values": {"owner": "hrt_qa", "qualifiedName": "hbase_table_aQTDg@cl1", "name": "hbase_table_aQTDg", "uri": "uris"}, "id": {"typeName": "hbase_table", "state": "ACTIVE", "version": 0, "id": "-74365953834994", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column_family", "state": "ACTIVE", "version": 0, "id": "-795512686603", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "qualifiedName": "hbase_table_aQTDg.profData.companyid@cl1", "name": "companyid", "owner": "hrt_qa", "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column", "state": "ACTIVE", "version": 0, "id": "-962353347235", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, {"traitNames": [], "typeName": "hbase_column", "traits": {}, "values": {"column_family": {"traitNames": [], "typeName": "hbase_column_family", "traits": {}, "values": {"qualifiedName": "hbase_table_aQTDg.profData@cl1", "name": "profData", "owner": "hrt_qa", "table": {"traitNames": [], "typeName": "hbase_table", "traits": {}, "values": {"owner": "hrt_qa", "qualifiedName": "hbase_table_aQTDg@cl1", "name": "hbase_table_aQTDg", "uri": "uris"}, "id": {"typeName": "hbase_table", "state": "ACTIVE", "version": 0, "id": "-74365953834994", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column_family", "state": "ACTIVE", "version": 0, "id": "-795512686603", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}, "qualifiedName": "hbase_table_aQTDg.profData.designation@cl1", "name": "designation", "owner": "hrt_qa", "isUnique": "False", "uri": "uris"}, "id": {"typeName": "hbase_column", "state": "ACTIVE", "version": 0, "id": "-468158960772", "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id"}, "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference"}], "type": "ENTITY_CREATE", "user": "hrt_qa"}, "version": {"version": "1.0.0"}}' | /usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list ctr-e129-1487033772569-11862-01-000010.hwx.site:6666 --topic ATLAS_HOOK --producer-property "security.protocol=SSL" --producer-property "ssl.truststore.location=/etc/security/clientKeys/all.jks" --producer-property "ssl.truststore.password=clientTrustStorePassword" --security-protocol SSL
> {noformat}
> application log snapshot
> {noformat}
> 2017-02-21 08:58:07,006 WARN - [NotificationHookConsumer thread-0:] ~ Error handling message: null (NotificationHookConsumer:315)
> 2017-02-21 08:58:07,006 INFO - [NotificationHookConsumer thread-0:] ~ Sleeping for 500 ms before retry (NotificationHookConsumer:317)
> 2017-02-21 08:58:07,507 WARN - [NotificationHookConsumer thread-0:] ~ Error handling message: null (NotificationHookConsumer:315)
> 2017-02-21 08:58:07,507 INFO - [NotificationHookConsumer thread-0:] ~ Sleeping for 500 ms before retry (NotificationHookConsumer:317)
> 2017-02-21 08:58:08,007 WARN - [NotificationHookConsumer thread-0:] ~ Error handling message: null (NotificationHookConsumer:315)
> 2017-02-21 08:58:08,007 INFO - [NotificationHookConsumer thread-0:] ~ Sleeping for 500 ms before retry (NotificationHookConsumer:317)
> 2017-02-21 08:58:08,508 WARN - [NotificationHookConsumer thread-0:] ~ Max retries exceeded for message [{Id='(type: hbase_table, id: <unassigned>)', traits=[], values={owner=hrt_qa, name=hbase_table_aQTDg, uri=uris, qualifiedName=hbase_table_aQTDg@cl1}}, {Id='(type: hbase_column_family, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.persData@cl1, name=persData, isUnique=False, uri=uris, table={Id='(type: hbase_table, id: <unassigned>)', traits=[], values={owner=hrt_qa, name=hbase_table_aQTDg, uri=uris, qualifiedName=hbase_table_aQTDg@cl1}}}}, {Id='(type: hbase_column, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.persData.name@cl1, column_family={Id='(type: hbase_column_family, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.persData@cl1, name=persData, isUnique=False, uri=uris, table={Id='(type: hbase_table, id: <unassigned>)', traits=[], values={owner=hrt_qa, name=hbase_table_aQTDg, uri=uris, qualifiedName=hbase_table_aQTDg@cl1}}}}, name=name, isUnique=False, uri=uris}}, {Id='(type: hbase_column, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.persData.age@cl1, column_family={Id='(type: hbase_column_family, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.persData@cl1, name=persData, isUnique=False, uri=uris, table={Id='(type: hbase_table, id: <unassigned>)', traits=[], values={owner=hrt_qa, name=hbase_table_aQTDg, uri=uris, qualifiedName=hbase_table_aQTDg@cl1}}}}, name=age, isUnique=False, uri=uris}}, {Id='(type: hbase_column_family, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.profData@cl1, name=profData, isUnique=False, uri=uris, table={Id='(type: hbase_table, id: <unassigned>)', traits=[], values={owner=hrt_qa, name=hbase_table_aQTDg, uri=uris, qualifiedName=hbase_table_aQTDg@cl1}}}}, {Id='(type: hbase_column, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.profData.companyid@cl1, column_family={Id='(type: hbase_column_family, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.profData@cl1, name=profData, isUnique=False, uri=uris, table={Id='(type: hbase_table, id: <unassigned>)', traits=[], values={owner=hrt_qa, name=hbase_table_aQTDg, uri=uris, qualifiedName=hbase_table_aQTDg@cl1}}}}, name=companyid, isUnique=False, uri=uris}}, {Id='(type: hbase_column, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.profData.designation@cl1, column_family={Id='(type: hbase_column_family, id: <unassigned>)', traits=[], values={owner=hrt_qa, qualifiedName=hbase_table_aQTDg.profData@cl1, name=profData, isUnique=False, uri=uris, table={Id='(type: hbase_table, id: <unassigned>)', traits=[], values={owner=hrt_qa, name=hbase_table_aQTDg, uri=uris, qualifiedName=hbase_table_aQTDg@cl1}}}}, name=designation, isUnique=False, uri=uris}}] (NotificationHookConsumer:324)
> java.lang.NullPointerException
> at org.apache.atlas.repository.converters.AtlasStructFormatConverter.fromV1ToV2(AtlasStructFormatConverter.java:160)
> at org.apache.atlas.repository.converters.AtlasEntityFormatConverter.fromV1ToV2(AtlasEntityFormatConverter.java:74)
> at org.apache.atlas.repository.converters.AtlasInstanceConverter.fromV1toV2Entity(AtlasInstanceConverter.java:222)
> at org.apache.atlas.repository.converters.AtlasInstanceConverter.toAtlasEntities(AtlasInstanceConverter.java:204)
> at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:256)
> at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:230)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)