You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@atlas.apache.org by sailaja ch <sa...@gmail.com> on 2017/09/20 16:47:11 UTC
Hive hook errors
I am using HDP 2.6.2.I created a table from both Hive CLI and beeline and
inserted a row with Atlas Hive hook enabled. From Hive CLI the insert goes
through fine without any errors but from beeline I see errors for a temp
table not found although insert itself succeeds and temp table shows up in
table listing. Do I need to do anything extra for beeline/HS2 connections
when Atlas is enabled ?
hive> create table test1(c1 int, c2 string);
OK
Time taken: 2.108 seconds
hive> insert into test1 values(1,'v');
Query ID = hive_20170920092542_d8ec0a4b-b88d-4d87-b59d-790f2eae6792
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1505852425742_0004)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED
KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 1 1 0 0 0
0
--------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 4.68 s
--------------------------------------------------------------------------------
Loading data to table default.test1
Table default.test1 stats: [numFiles=1, numRows=1, totalSize=4,
rawDataSize=3]
OK
Time taken: 6.7 seconds
hive> select * from test1;
OK
1 v
Time taken: 0.112 seconds, Fetched: 1 row(s)
hive>
Beeline version 1.2.1000.2.6.2.0-133 by Apache Hive
0: jdbc:hive2://> create table s1(c1 int,c2 string);
OK
No rows affected (2.345 seconds)
0: jdbc:hive2://> insert into s1 values(1,'v');
17/09/20 09:28:24 [main]: ERROR hdfs.KeyProviderCache: Could not find uri
with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
Query ID = hive_20170920092824_14bfdec4-dfe8-4b97-b5be-0af74de055ce
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id
application_1505852425742_0005)
Loading data to table default.s1
Table default.s1 stats: [numFiles=1, numRows=1, totalSize=4, rawDataSize=3]
OK
No rows affected (498.295 seconds)
0: jdbc:hive2://>
0: jdbc:hive2://> 17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive:
Table values__tmp__table__1 not found: default.values__tmp__table__1 table
not found
17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed
due to error
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.atlas.hook.AtlasHookException:
HiveHook.registerProcess() failed.
at
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:701)
at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268)
at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:83)
at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
... 6 more
Caused by: org.apache.atlas.hook.AtlasHookException:
HiveHook.processHiveEntity() failed.
at
org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:731)
at
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:668)
... 12 more
Caused by: org.apache.atlas.hook.AtlasHookException:
HiveHook.createOrUpdateEntities() failed.
at
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:597)
at
org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:711)
... 13 more
Caused by: org.apache.atlas.hook.AtlasHookException:
HiveHook.createOrUpdateEntities() failed.
at
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:589)
at
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:595)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table
not found values__tmp__table__1
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1216)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1186)
at
org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:568)
... 15 more
0: jdbc:hive2://> select * from s1;
OK
+--------+--------+--+
| s1.c1 | s1.c2 |
+--------+--------+--+
| 1 | v |
+--------+--------+--+
1 row selected (0.259 seconds)
0: jdbc:hive2://>
0: jdbc:hive2://> show tables;
OK
+------------------------+--+
| tab_name |
+------------------------+--+
| s1 |
| test1 |
| values__tmp__table__1 |
+------------------------+--+
3 rows selected (0.075 seconds)
Sailaja
Re: Hive hook errors
Posted by sailaja ch <sa...@gmail.com>.
Filed https://issues.apache.org/jira/browse/ATLAS-2157 for the issue.
Thanks for the quick response
Sailaja
On Wed, Sep 20, 2017 at 6:03 PM, Madhan Neethiraj <ma...@apache.org> wrote:
> Sailaja,
>
>
>
> This error message is due to an unhandled exception in Hive hook, and
> happens when temporary tables are involved. This error message can be
> ignored.
>
>
>
> Can you please file a JIRA in Apache for this issue? We will have the fix
> shortly.
>
>
>
> Thanks,
>
> Madhan
>
>
>
>
>
> *From: *sailaja ch <sa...@gmail.com>
> *Reply-To: *"user@atlas.apache.org" <us...@atlas.apache.org>
> *Date: *Wednesday, September 20, 2017 at 9:47 AM
> *To: *"user@atlas.apache.org" <us...@atlas.apache.org>
> *Subject: *Hive hook errors
>
>
>
> I am using HDP 2.6.2.I created a table from both Hive CLI and beeline and
> inserted a row with Atlas Hive hook enabled. From Hive CLI the insert goes
> through fine without any errors but from beeline I see errors for a temp
> table not found although insert itself succeeds and temp table shows up in
> table listing. Do I need to do anything extra for beeline/HS2 connections
> when Atlas is enabled ?
>
>
>
> hive> create table test1(c1 int, c2 string);
>
> OK
>
> Time taken: 2.108 seconds
>
> hive> insert into test1 values(1,'v');
>
> Query ID = hive_20170920092542_d8ec0a4b-b88d-4d87-b59d-790f2eae6792
>
> Total jobs = 1
>
> Launching Job 1 out of 1
>
> Status: Running (Executing on YARN cluster with App id
> application_1505852425742_0004)
>
>
>
> ------------------------------------------------------------
> --------------------
>
> VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED
> KILLED
>
> ------------------------------------------------------------
> --------------------
>
> Map 1 .......... SUCCEEDED 1 1 0 0 0
> 0
>
> ------------------------------------------------------------
> --------------------
>
> VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 4.68 s
>
> ------------------------------------------------------------
> --------------------
>
> Loading data to table default.test1
>
> Table default.test1 stats: [numFiles=1, numRows=1, totalSize=4,
> rawDataSize=3]
>
> OK
>
> Time taken: 6.7 seconds
>
> hive> select * from test1;
>
> OK
>
> 1 v
>
> Time taken: 0.112 seconds, Fetched: 1 row(s)
>
> hive>
>
>
>
> Beeline version 1.2.1000.2.6.2.0-133 by Apache Hive
>
> 0: jdbc:hive2://> create table s1(c1 int,c2 string);
>
> OK
>
> No rows affected (2.345 seconds)
>
>
>
> 0: jdbc:hive2://> insert into s1 values(1,'v');
>
> 17/09/20 09:28:24 [main]: ERROR hdfs.KeyProviderCache: Could not find uri
> with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
>
> Query ID = hive_20170920092824_14bfdec4-dfe8-4b97-b5be-0af74de055ce
>
> Total jobs = 1
>
> Launching Job 1 out of 1
>
> Status: Running (Executing on YARN cluster with App id
> application_1505852425742_0005)
>
>
>
> Loading data to table default.s1
>
> Table default.s1 stats: [numFiles=1, numRows=1, totalSize=4, rawDataSize=3]
>
> OK
>
> No rows affected (498.295 seconds)
>
> 0: jdbc:hive2://>
>
> 0: jdbc:hive2://> 17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive:
> Table values__tmp__table__1 not found: default.values__tmp__table__1 table
> not found
>
> 17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed
> due to error
>
> java.lang.reflect.UndeclaredThrowableException
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1884)
>
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195)
>
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
>
> at java.lang.Thread.run(Thread.java:745)
>
> Caused by: org.apache.atlas.hook.AtlasHookException:
> HiveHook.registerProcess() failed.
>
> at org.apache.atlas.hive.hook.HiveHook.registerProcess(
> HiveHook.java:701)
>
> at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268)
>
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.
> java:83)
>
> at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198)
>
> at java.security.AccessController.doPrivileged(Native Method)
>
> at javax.security.auth.Subject.doAs(Subject.java:422)
>
> at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1866)
>
> ... 6 more
>
> Caused by: org.apache.atlas.hook.AtlasHookException:
> HiveHook.processHiveEntity() failed.
>
> at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(
> HiveHook.java:731)
>
> at org.apache.atlas.hive.hook.HiveHook.registerProcess(
> HiveHook.java:668)
>
> ... 12 more
>
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities()
> failed.
>
> at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(
> HiveHook.java:597)
>
> at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(
> HiveHook.java:711)
>
> ... 13 more
>
> Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities()
> failed.
>
> at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(
> HiveHook.java:589)
>
> at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(
> HiveHook.java:595)
>
> ... 14 more
>
> Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException:
> Table not found values__tmp__table__1
>
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.
> java:1216)
>
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.
> java:1186)
>
> at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(
> HiveHook.java:568)
>
> ... 15 more
>
> 0: jdbc:hive2://> select * from s1;
>
> OK
>
> +--------+--------+--+
>
> | s1.c1 | s1.c2 |
>
> +--------+--------+--+
>
> | 1 | v |
>
> +--------+--------+--+
>
> 1 row selected (0.259 seconds)
>
> 0: jdbc:hive2://>
>
> 0: jdbc:hive2://> show tables;
>
> OK
>
> +------------------------+--+
>
> | tab_name |
>
> +------------------------+--+
>
> | s1 |
>
> | test1 |
>
> | values__tmp__table__1 |
>
> +------------------------+--+
>
> 3 rows selected (0.075 seconds)
>
>
>
> Sailaja
>
Re: Hive hook errors
Posted by Madhan Neethiraj <ma...@apache.org>.
Sailaja,
This error message is due to an unhandled exception in Hive hook, and happens when temporary tables are involved. This error message can be ignored.
Can you please file a JIRA in Apache for this issue? We will have the fix shortly.
Thanks,
Madhan
From: sailaja ch <sa...@gmail.com>
Reply-To: "user@atlas.apache.org" <us...@atlas.apache.org>
Date: Wednesday, September 20, 2017 at 9:47 AM
To: "user@atlas.apache.org" <us...@atlas.apache.org>
Subject: Hive hook errors
I am using HDP 2.6.2.I created a table from both Hive CLI and beeline and inserted a row with Atlas Hive hook enabled. From Hive CLI the insert goes through fine without any errors but from beeline I see errors for a temp table not found although insert itself succeeds and temp table shows up in table listing. Do I need to do anything extra for beeline/HS2 connections when Atlas is enabled ?
hive> create table test1(c1 int, c2 string);
OK
Time taken: 2.108 seconds
hive> insert into test1 values(1,'v');
Query ID = hive_20170920092542_d8ec0a4b-b88d-4d87-b59d-790f2eae6792
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1505852425742_0004)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 1 1 0 0 0 0
--------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 4.68 s
--------------------------------------------------------------------------------
Loading data to table default.test1
Table default.test1 stats: [numFiles=1, numRows=1, totalSize=4, rawDataSize=3]
OK
Time taken: 6.7 seconds
hive> select * from test1;
OK
1 v
Time taken: 0.112 seconds, Fetched: 1 row(s)
hive>
Beeline version 1.2.1000.2.6.2.0-133 by Apache Hive
0: jdbc:hive2://> create table s1(c1 int,c2 string);
OK
No rows affected (2.345 seconds)
0: jdbc:hive2://> insert into s1 values(1,'v');
17/09/20 09:28:24 [main]: ERROR hdfs.KeyProviderCache: Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!
Query ID = hive_20170920092824_14bfdec4-dfe8-4b97-b5be-0af74de055ce
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1505852425742_0005)
Loading data to table default.s1
Table default.s1 stats: [numFiles=1, numRows=1, totalSize=4, rawDataSize=3]
OK
No rows affected (498.295 seconds)
0: jdbc:hive2://>
0: jdbc:hive2://> 17/09/20 09:36:42 [Atlas Logger 1]: ERROR metadata.Hive: Table values__tmp__table__1 not found: default.values__tmp__table__1 table not found
17/09/20 09:36:43 [Atlas Logger 1]: ERROR hook.HiveHook: Atlas hook failed due to error
java.lang.reflect.UndeclaredThrowableException
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:195)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.registerProcess() failed.
at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:701)
at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:268)
at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:83)
at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:198)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
... 6 more
Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.processHiveEntity() failed.
at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:731)
at org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:668)
... 12 more
Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities() failed.
at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:597)
at org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:711)
... 13 more
Caused by: org.apache.atlas.hook.AtlasHookException: HiveHook.createOrUpdateEntities() failed.
at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:589)
at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:595)
... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found values__tmp__table__1
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1216)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1186)
at org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.java:568)
... 15 more
0: jdbc:hive2://> select * from s1;
OK
+--------+--------+--+
| s1.c1 | s1.c2 |
+--------+--------+--+
| 1 | v |
+--------+--------+--+
1 row selected (0.259 seconds)
0: jdbc:hive2://>
0: jdbc:hive2://> show tables;
OK
+------------------------+--+
| tab_name |
+------------------------+--+
| s1 |
| test1 |
| values__tmp__table__1 |
+------------------------+--+
3 rows selected (0.075 seconds)
Sailaja