You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Sowjanya Kakarala <so...@agrible.com> on 2018/04/26 18:49:23 UTC
java.io.IOException: NoSuchObjectException
Hi everyone,
I have two questions as below, any help is appreciable. Thanks in advance.
1. I am trying to execute a sqoop command in aws cluster, where I created
the table in hive and then ran this command. But I am getting the error
where it says there is no database or table.
*Command*:
[hadoop@ip-172-31-35-8 ~]$ sqoop import --connect
"jdbc:postgresql://hostname/dbname" --query 'select id, data from sometable
where $CONDITIONS' --username 'uname' --password 'pwd' --hcatalog-database
'dbname' --hcatalog-table 'tblname' --hive-partition-key 'partitionkey'
--hive-partition-value 'value' -m 4 --split-by 'cell_id'
*HIVE*:
hive> use dbname;
OK
Time taken: 0.239 seconds
hive> show tables;
OK
tblname
Time taken: 0.346 seconds, Fetched: 7 row(s)
*LOG/Error*:
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will
fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/
slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/
log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/04/26 18:20:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
18/04/26 18:20:41 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
18/04/26 18:20:41 INFO manager.SqlManager: Using default fetchSize of 1000
18/04/26 18:20:41 INFO tool.CodeGenTool: Beginning code generation
18/04/26 18:20:42 INFO manager.SqlManager: Executing SQL statement: select
id, data from sometable where (1 = 0)
18/04/26 18:20:42 INFO manager.SqlManager: Executing SQL statement: select
id, data from sometable where (1 = 0)
18/04/26 18:20:42 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
/usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-hadoop/compile/cabf738703e310c5416ea4225006272e/QueryResult.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/04/26 18:20:44 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-hadoop/compile/cabf738703e310c5416ea4225006272e/QueryResult.jar
18/04/26 18:20:44 INFO mapreduce.ImportJobBase: Beginning query import.
18/04/26 18:20:44 INFO Configuration.deprecation: mapred.jar is deprecated.
Instead, use mapreduce.job.jar
18/04/26 18:20:44 INFO hcat.SqoopHCatUtilities: Configuring HCatalog for
import job
18/04/26 18:20:44 INFO hcat.SqoopHCatUtilities: Configuring HCatalog
specific details for job
18/04/26 18:20:44 INFO manager.SqlManager: Executing SQL statement: select
id, data from sometable where (1 = 0)
18/04/26 18:20:44 INFO manager.SqlManager: Executing SQL statement: select
id, data from sometable where (1 = 0)
18/04/26 18:20:44 INFO hcat.SqoopHCatUtilities: Database column names
projected : [cell_id, data]
18/04/26 18:20:44 INFO hcat.SqoopHCatUtilities: Database column name - info
map :
data : [Type : 8,Precision : 17,Scale : 17]
cell_id : [Type : 4,Precision : 10,Scale : 0]
18/04/26 18:20:44 INFO conf.HiveConf: Found configuration file
file:/etc/hive/conf.dist/hive-site.xml
18/04/26 18:20:45 INFO common.HiveClientCache: Initializing cache:
eviction-timeout=120 initial-capacity=50 maximum-capacity=50
18/04/26 18:20:45 INFO hive.metastore: Trying to connect to metastore with
URI thrift://ip-172-31-35-8.us-west-2.compute.internal:9083
18/04/26 18:20:45 INFO hive.metastore: Opened a connection to metastore,
current connections: 1
18/04/26 18:20:45 INFO hive.metastore: Connected to metastore.
18/04/26 18:20:45 ERROR tool.ImportTool: Encountered IOException running
import job: java.io.IOException:
NoSuchObjectException(message:monolith.nexrad_cell_4km_p
table not found)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.
setInput(HCatInputFormat.java:97)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.
setInput(HCatInputFormat.java:51)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(
SqoopHCatUtilities.java:343)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.
configureImportOutputFormat(SqoopHCatUtilities.java:783)
at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(
ImportJobBase.java:98)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(
ImportJobBase.java:259)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:729)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: NoSuchObjectException(message:dbname.tblname table not found)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_
req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java:
55064)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_
req_result$get_table_req_resultStandardScheme.read(ThriftHiveMetastore.java:
55032)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_
req_result.read(ThriftHiveMetastore.java:54963)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.
recv_get_table_req(ThriftHiveMetastore.java:1563)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.
get_table_req(ThriftHiveMetastore.java:1550)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
getTable(HiveMetaStoreClient.java:1344)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(
RetryingMetaStoreClient.java:169)
at com.sun.proxy.$Proxy5.getTable(Unknown Source)
at org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:180)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(
InitializeInput.java:105)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.
setInput(InitializeInput.java:88)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.
setInput(HCatInputFormat.java:95)
... 14 more
18/04/26 18:20:45 INFO hive.metastore: Closed a connection to metastore,
current connections: 0
====================================================================================================================
2. I tried a simple hcat command where it gave me below error: I tried
checking in hive-site.xml and there was this java class file mentioned as
follows:
<property>
<name>hive.metastore.client.factory.class</name>
<value>com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory</value>
</property>
*Command*:hcat -e "show tables;"
*ERROR*:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/
slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/
log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/04/26 18:15:51 INFO conf.HiveConf: Found configuration file
file:/etc/hive/conf.dist/hive-site.xml
18/04/26 18:15:53 INFO cli.HCatCli: Forcing hive.execution.engine to mr
18/04/26 18:15:54 INFO session.SessionState: Created HDFS directory:
/tmp/hive/root/29db66cd-9707-4b84-85cd-0c33c0f46f74
18/04/26 18:15:54 INFO session.SessionState: Created local directory:
/tmp/root/29db66cd-9707-4b84-85cd-0c33c0f46f74
18/04/26 18:15:54 INFO session.SessionState: Created HDFS directory:
/tmp/hive/root/29db66cd-9707-4b84-85cd-0c33c0f46f74/_tmp_space.db
18/04/26 18:15:54 INFO ql.Driver: Compiling command(queryId=root_
20180426181554_952464e6-99cf-446f-b196-4ac5292a1317): show tables
18/04/26 18:15:55 ERROR metadata.Hive: Unable to instantiate a metastore
client factory com.amazonaws.glue.catalog.metastore.
AWSGlueDataCatalogHiveClientFactory due to: java.lang.ClassNotFoundException:
Class com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found
java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found
at org.apache.hadoop.conf.Configuration.getClassByName(
Configuration.java:2273)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClientFactory(
Hive.java:3629)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.
java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.
java:232)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:289)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.createHiveDB(
BaseSemanticAnalyzer.java:236)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.<
init>(DDLSemanticAnalyzer.java:232)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzerFactory.
get(SemanticAnalyzerFactory.java:293)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:484)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
18/04/26 18:15:55 WARN metadata.Hive: Failed to register all functions.
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3933)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.
java:232)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:289)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.createHiveDB(
BaseSemanticAnalyzer.java:236)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.<
init>(DDLSemanticAnalyzer.java:232)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzerFactory.
get(SemanticAnalyzerFactory.java:293)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:484)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: MetaException(message:Unable to instantiate a metastore client
factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClientFactory(
Hive.java:3637)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.
java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
... 24 more
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
18/04/26 18:15:55 ERROR ql.Driver: FAILED: SemanticException
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
org.apache.hadoop.hive.ql.parse.SemanticException:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.createHiveDB(
BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.<
init>(DDLSemanticAnalyzer.java:232)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzerFactory.
get(SemanticAnalyzerFactory.java:293)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:484)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.
java:237)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:289)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.createHiveDB(
BaseSemanticAnalyzer.java:236)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3933)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.
java:232)
... 22 more
Caused by: MetaException(message:Unable to instantiate a metastore client
factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClientFactory(
Hive.java:3637)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.
java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
... 24 more
18/04/26 18:15:55 ERROR metadata.Hive: Unable to instantiate a metastore
client factory com.amazonaws.glue.catalog.metastore.
AWSGlueDataCatalogHiveClientFactory due to: java.lang.ClassNotFoundException:
Class com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found
java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found
at org.apache.hadoop.conf.Configuration.getClassByName(
Configuration.java:2273)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClientFactory(
Hive.java:3629)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.
java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.
java:232)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:355)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:351)
at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(
Driver.java:683)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
18/04/26 18:15:55 WARN metadata.Hive: Failed to register all functions.
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3933)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.
java:232)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:355)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:351)
at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(
Driver.java:683)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: MetaException(message:Unable to instantiate a metastore client
factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClientFactory(
Hive.java:3637)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.
java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
... 23 more
18/04/26 18:15:55 WARN ql.Driver: Caught exception attempting to write
metadata call information org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.
java:237)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:355)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:351)
at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(
Driver.java:683)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3933)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.
java:232)
... 21 more
Caused by: MetaException(message:Unable to instantiate a metastore client
factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClientFactory(
Hive.java:3637)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.
java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
... 23 more
18/04/26 18:15:55 INFO ql.Driver: Completed compiling command(queryId=root_
20180426181554_952464e6-99cf-446f-b196-4ac5292a1317); Time taken: 1.185
seconds
18/04/26 18:15:55 INFO session.SessionState: Deleted directory:
/tmp/hive/root/29db66cd-9707-4b84-85cd-0c33c0f46f74 on fs with scheme hdfs
18/04/26 18:15:55 INFO session.SessionState: Deleted directory:
/tmp/root/29db66cd-9707-4b84-85cd-0c33c0f46f74 on fs with scheme file
18/04/26 18:15:55 ERROR metadata.Hive: Unable to instantiate a metastore
client factory com.amazonaws.glue.catalog.metastore.
AWSGlueDataCatalogHiveClientFactory due to: java.lang.ClassNotFoundException:
Class com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found
java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.
metastore.AWSGlueDataCatalogHiveClientFactory not found
at org.apache.hadoop.conf.Configuration.getClassByName(
Configuration.java:2273)
Thanks
Sowjanya
Fwd: java.io.IOException: NoSuchObjectException
Posted by Sowjanya Kakarala <so...@agrible.com>.
Hi everyone,
I have two questions as below, any help is appreciable. Thanks in advance.
1. I am trying to execute a sqoop command in aws cluster, where I created
the table in hive and then ran this command. But I am getting the error
where it says there is no database or table.
*Command*:
[hadoop@ip-172-31-35-8 ~]$ sqoop import --connect
"jdbc:postgresql://hostname/dbname" --query 'select id, data from sometable
where $CONDITIONS' --username 'uname' --password 'pwd' --hcatalog-database
'dbname' --hcatalog-table 'tblname' --hive-partition-key 'partitionkey'
--hive-partition-value 'value' -m 4 --split-by 'cell_id'
*HIVE*:
hive> use dbname;
OK
Time taken: 0.239 seconds
hive> show tables;
OK
tblname
Time taken: 0.346 seconds, Fetched: 7 row(s)
*LOG/Error*:
Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will
fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/
slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/lo
g4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/04/26 18:20:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
18/04/26 18:20:41 WARN tool.BaseSqoopTool: Setting your password on the
command-line is insecure. Consider using -P instead.
18/04/26 18:20:41 INFO manager.SqlManager: Using default fetchSize of 1000
18/04/26 18:20:41 INFO tool.CodeGenTool: Beginning code generation
18/04/26 18:20:42 INFO manager.SqlManager: Executing SQL statement: select
id, data from sometable where (1 = 0)
18/04/26 18:20:42 INFO manager.SqlManager: Executing SQL statement: select
id, data from sometable where (1 = 0)
18/04/26 18:20:42 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
/usr/lib/hadoop-mapreduce
Note: /tmp/sqoop-hadoop/compile/cabf738703e310c5416ea4225006272e/QueryResult.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
18/04/26 18:20:44 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-hadoop/compile/cabf738703e310c5416ea4225006272e/QueryResult.jar
18/04/26 18:20:44 INFO mapreduce.ImportJobBase: Beginning query import.
18/04/26 18:20:44 INFO Configuration.deprecation: mapred.jar is deprecated.
Instead, use mapreduce.job.jar
18/04/26 18:20:44 INFO hcat.SqoopHCatUtilities: Configuring HCatalog for
import job
18/04/26 18:20:44 INFO hcat.SqoopHCatUtilities: Configuring HCatalog
specific details for job
18/04/26 18:20:44 INFO manager.SqlManager: Executing SQL statement: select
id, data from sometable where (1 = 0)
18/04/26 18:20:44 INFO manager.SqlManager: Executing SQL statement: select
id, data from sometable where (1 = 0)
18/04/26 18:20:44 INFO hcat.SqoopHCatUtilities: Database column names
projected : [cell_id, data]
18/04/26 18:20:44 INFO hcat.SqoopHCatUtilities: Database column name - info
map :
data : [Type : 8,Precision : 17,Scale : 17]
cell_id : [Type : 4,Precision : 10,Scale : 0]
18/04/26 18:20:44 INFO conf.HiveConf: Found configuration file
file:/etc/hive/conf.dist/hive-site.xml
18/04/26 18:20:45 INFO common.HiveClientCache: Initializing cache:
eviction-timeout=120 initial-capacity=50 maximum-capacity=50
18/04/26 18:20:45 INFO hive.metastore: Trying to connect to metastore with
URI thrift://ip-172-31-35-8.us-west-2.compute.internal:9083
18/04/26 18:20:45 INFO hive.metastore: Opened a connection to metastore,
current connections: 1
18/04/26 18:20:45 INFO hive.metastore: Connected to metastore.
18/04/26 18:20:45 ERROR tool.ImportTool: Encountered IOException running
import job: java.io.IOException:
NoSuchObjectException(message:monolith.nexrad_cell_4km_p
table not found)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(
HCatInputFormat.java:97)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(
HCatInputFormat.java:51)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configure
HCat(SqoopHCatUtilities.java:343)
at org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configure
ImportOutputFormat(SqoopHCatUtilities.java:783)
at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputForm
at(ImportJobBase.java:98)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJob
Base.java:259)
at org.apache.sqoop.manager.SqlManager.importQuery(SqlManager.java:729)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:499)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: NoSuchObjectException(message:dbname.tblname table not found)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$
get_table_req_result$get_table_req_resultStandardScheme
.read(ThriftHiveMetastore.java:55064)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$
get_table_req_result$get_table_req_resultStandardScheme
.read(ThriftHiveMetastore.java:55032)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$
get_table_req_result.read(ThriftHiveMetastore.java:54963)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$
Client.recv_get_table_req(ThriftHiveMetastore.java:1563)
at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$
Client.get_table_req(ThriftHiveMetastore.java:1550)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTabl
e(HiveMetaStoreClient.java:1344)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.inv
oke(RetryingMetaStoreClient.java:169)
at com.sun.proxy.$Proxy5.getTable(Unknown Source)
at org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:180)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJ
obInfo(InitializeInput.java:105)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(
InitializeInput.java:88)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(
HCatInputFormat.java:95)
... 14 more
18/04/26 18:20:45 INFO hive.metastore: Closed a connection to metastore,
current connections: 0
============================================================
========================================================
2. I tried a simple hcat command where it gave me below error: I tried
checking in hive-site.xml and there was this java class file mentioned as
follows:
<property>
<name>hive.metastore.client.factory.class</name>
<value>com.amazonaws.glue.catalog.metastore.
AWSGlueDataCatalogHiveClientFactory</value>
</property>
*Command*:hcat -e "show tables;"
*ERROR*:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/
slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/lo
g4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
18/04/26 18:15:51 INFO conf.HiveConf: Found configuration file
file:/etc/hive/conf.dist/hive-site.xml
18/04/26 18:15:53 INFO cli.HCatCli: Forcing hive.execution.engine to mr
18/04/26 18:15:54 INFO session.SessionState: Created HDFS directory:
/tmp/hive/root/29db66cd-9707-4b84-85cd-0c33c0f46f74
18/04/26 18:15:54 INFO session.SessionState: Created local directory:
/tmp/root/29db66cd-9707-4b84-85cd-0c33c0f46f74
18/04/26 18:15:54 INFO session.SessionState: Created HDFS directory:
/tmp/hive/root/29db66cd-9707-4b84-85cd-0c33c0f46f74/_tmp_space.db
18/04/26 18:15:54 INFO ql.Driver: Compiling command(queryId=root_201804261
81554_952464e6-99cf-446f-b196-4ac5292a1317): show tables
18/04/26 18:15:55 ERROR metadata.Hive: Unable to instantiate a metastore
client factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found
java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.met
astore.AWSGlueDataCatalogHiveClientFactory not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configur
ation.java:2273)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
tFactory(Hive.java:3629)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
t(Hive.java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctions
Once(Hive.java:232)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:289)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.createH
iveDB(BaseSemanticAnalyzer.java:236)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.<init>(
DDLSemanticAnalyzer.java:232)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzerFactory.get(
SemanticAnalyzerFactory.java:293)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:484)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
18/04/26 18:15:55 WARN metadata.Hive: Failed to register all functions.
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3933)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctions
Once(Hive.java:232)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:289)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.createH
iveDB(BaseSemanticAnalyzer.java:236)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.<init>(
DDLSemanticAnalyzer.java:232)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzerFactory.get(
SemanticAnalyzerFactory.java:293)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:484)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: MetaException(message:Unable to instantiate a metastore client
factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
tFactory(Hive.java:3637)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
t(Hive.java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
... 24 more
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
18/04/26 18:15:55 ERROR ql.Driver: FAILED: SemanticException
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
org.apache.hadoop.hive.ql.parse.SemanticException:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.createH
iveDB(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.<init>(
DDLSemanticAnalyzer.java:232)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzerFactory.get(
SemanticAnalyzerFactory.java:293)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:484)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctions
Once(Hive.java:237)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:289)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.createH
iveDB(BaseSemanticAnalyzer.java:236)
... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3933)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctions
Once(Hive.java:232)
... 22 more
Caused by: MetaException(message:Unable to instantiate a metastore client
factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
tFactory(Hive.java:3637)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
t(Hive.java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
... 24 more
18/04/26 18:15:55 ERROR metadata.Hive: Unable to instantiate a metastore
client factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found
java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.met
astore.AWSGlueDataCatalogHiveClientFactory not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configur
ation.java:2273)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
tFactory(Hive.java:3629)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
t(Hive.java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctions
Once(Hive.java:232)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:355)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:351)
at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx
(Driver.java:683)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
18/04/26 18:15:55 WARN metadata.Hive: Failed to register all functions.
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3933)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctions
Once(Hive.java:232)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:355)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:351)
at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx
(Driver.java:683)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: MetaException(message:Unable to instantiate a metastore client
factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
tFactory(Hive.java:3637)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
t(Hive.java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
... 23 more
18/04/26 18:15:55 WARN ql.Driver: Caught exception attempting to write
metadata call information org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctions
Once(Hive.java:237)
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:390)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:333)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:313)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:355)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:351)
at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx
(Driver.java:683)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Unable to instantiate a metastore client factory
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3933)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:249)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctions
Once(Hive.java:232)
... 21 more
Caused by: MetaException(message:Unable to instantiate a metastore client
factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
tFactory(Hive.java:3637)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClien
t(Hive.java:3619)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3688)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3668)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3930)
... 23 more
18/04/26 18:15:55 INFO ql.Driver: Completed compiling
command(queryId=root_20180426181554_952464e6-99cf-446f-b196-4ac5292a1317);
Time taken: 1.185 seconds
18/04/26 18:15:55 INFO session.SessionState: Deleted directory:
/tmp/hive/root/29db66cd-9707-4b84-85cd-0c33c0f46f74 on fs with scheme hdfs
18/04/26 18:15:55 INFO session.SessionState: Deleted directory:
/tmp/root/29db66cd-9707-4b84-85cd-0c33c0f46f74 on fs with scheme file
18/04/26 18:15:55 ERROR metadata.Hive: Unable to instantiate a metastore
client factory com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
due to: java.lang.ClassNotFoundException: Class
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory
not found
java.lang.ClassNotFoundException: Class com.amazonaws.glue.catalog.met
astore.AWSGlueDataCatalogHiveClientFactory not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configur
ation.java:2273)
Thanks
Sowjanya