[jira] [Created] (ATLAS-3475) Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable

["Sarath Subramanian (Jira)" <ji...@apache.org>]

Sarath Subramanian created ATLAS-3475:
-----------------------------------------

             Summary: Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable
                 Key: ATLAS-3475
                 URL: https://issues.apache.org/jira/browse/ATLAS-3475
             Project: Atlas
          Issue Type: Bug
          Components:  atlas-core
    Affects Versions: 2.0.0
            Reporter: Sarath Subramanian
            Assignee: Sarath Subramanian
             Fix For: 2.1.0


While running CTAS query via spark.sql, Atlas is throwing below error

{code:java}
org.apache.atlas.exception.AtlasBaseException: Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceableorg.apache.atlas.exception.AtlasBaseException: Invalid instance creation/updation parameters passed : hive_process.qualifiedName: mandatory attribute value missing in type Referenceable at org.apache.atlas.repository.store.graph.v2.AtlasEntityGraphDiscoveryV2.validateAndNormalize(AtlasEntityGraphDiscoveryV2.java:103) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.preCreateOrUpdate(AtlasEntityStoreV2.java:938) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:748) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2.createOrUpdate(AtlasEntityStoreV2.java:279) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$FastClassBySpringCGLIB$$6861dca9.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:82) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671) at org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2$$EnhancerBySpringCGLIB$$91e799f5.createOrUpdate(<generated>) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.createOrUpdate(NotificationHookConsumer.java:802) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:700) at org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.doWork(NotificationHookConsumer.java:521) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:89) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) \{code}
The above error comes even in a newly created cluster, where there are (were) no shell entities present.

the error message is mostly harmless since the entities are getting created properly. Shell entities too are getting resolved correctly.

But in the metrics UI, each CTAS query shows up as a failure under ATLAS_HOOK-0 failed messages 

Steps to reproduce:
 # spark-shell
 # spark.sql("create table t1(name string)")
 # spark.sql("create table t2 as select * from t1")



--
This message was sent by Atlassian Jira
(v8.3.4#803005)