You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Aegeaner (JIRA)" <ji...@apache.org> on 2017/11/29 10:18:00 UTC
[jira] [Created] (HIVE-18177) Error creating temporary staging
folder on HDFS when creating Hive table
Aegeaner created HIVE-18177:
-------------------------------
Summary: Error creating temporary staging folder on HDFS when creating Hive table
Key: HIVE-18177
URL: https://issues.apache.org/jira/browse/HIVE-18177
Project: Hive
Issue Type: Bug
Components: Parser
Affects Versions: 2.3.1
Reporter: Aegeaner
When creating a table in hive using statement:
{code:java}
create database if not exists ${DB};
use ${DB};
drop table if exists date_dim;
create table date_dim
stored as ${FILE}
as select * from ${SOURCE}.date_dim;
{code}
We got this exception stackļ¼
{code:java}
017-11-29T17:32:47,646 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] parse.CalcitePlanner: Completed phase 1 of Semantic Analysis
2017-11-29T17:32:47,646 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] parse.CalcitePlanner: Get metadata for source tables
2017-11-29T17:32:47,646 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] metastore.HiveMetaStore: 0: get_table : db=tpcds_text_2 tbl=date_dim
2017-11-29T17:32:47,647 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_table : db=tpcds_text_2 tbl=date_dim
2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] parse.CalcitePlanner: Get metadata for subqueries
2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] parse.CalcitePlanner: Get metadata for destination tables
2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] metastore.HiveMetaStore: 0: get_database: tpcds_bin_partitioned_orc_2
2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_database: tpcds_bin_partitioned_orc_2
2017-11-29T17:32:48,308 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] common.FileUtils: Creating directory if it doesn't exist: hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db/.hive-staging_hive_2017-11-29_17-32-47_541_2322222506518783479-1
2017-11-29T17:32:48,330 ERROR [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] parse.CalcitePlanner: org.apache.hadoop.hive.ql.parse.SemanticException: 0:0 Error creating temporary folder on: hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db. Error encountered near token 'TOK_TMP_FILE'
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2211)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1934)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:11080)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11133)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336)
at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:474)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:490)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: java.lang.RuntimeException: Cannot create staging directory 'hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db/.hive-staging_hive_2017-11-29_17-32-47_541_2322222506518783479-1': Invalid host name: local host is: (unknown); destination host is: "ns-offline":8020; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost
at org.apache.hadoop.hive.ql.Context.getStagingDir(Context.java:374)
at org.apache.hadoop.hive.ql.Context.getExtTmpPathRelTo(Context.java:632)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2208)
... 25 more
Caused by: java.net.UnknownHostException: Invalid host name: local host is: (unknown); destination host is: "ns-offline":8020; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:801)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:744)
at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:445)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1522)
at org.apache.hadoop.ipc.Client.call(Client.java:1373)
at org.apache.hadoop.ipc.Client.call(Client.java:1337)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy36.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:787)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:398)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
at com.sun.proxy.$Proxy37.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1700)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1436)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1433)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1433)
at org.apache.hadoop.hive.common.FileUtils.mkdir(FileUtils.java:528)
at org.apache.hadoop.hive.ql.Context.getStagingDir(Context.java:366)
... 27 more
Caused by: java.net.UnknownHostException
... 52 more
2017-11-29T17:32:48,331 ERROR [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main] ql.Driver: FAILED: SemanticException 0:0 Error creating temporary folder on: hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db. Error encountered near token 'TOK_TMP_FILE'
org.apache.hadoop.hive.ql.parse.SemanticException: 0:0 Error creating temporary folder on: hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db. Error encountered near token 'TOK_TMP_FILE'
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2211)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1934)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:11080)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11133)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336)
at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:474)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:490)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: java.lang.RuntimeException: Cannot create staging directory 'hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db/.hive-staging_hive_2017-11-29_17-32-47_541_2322222506518783479-1': Invalid host name: local host is: (unknown); destination host is: "ns-offline":8020; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost
at org.apache.hadoop.hive.ql.Context.getStagingDir(Context.java:374)
at org.apache.hadoop.hive.ql.Context.getExtTmpPathRelTo(Context.java:632)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2208)
... 25 more
Caused by: java.net.UnknownHostException: Invalid host name: local host is: (unknown); destination host is: "ns-offline":8020; java.net.UnknownHostException; For more details see: http://wiki.apache.org/hadoop/UnknownHost
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:801)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:744)
at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:445)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1522)
at org.apache.hadoop.ipc.Client.call(Client.java:1373)
at org.apache.hadoop.ipc.Client.call(Client.java:1337)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy36.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:787)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:398)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
at com.sun.proxy.$Proxy37.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1700)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1436)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1433)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1433)
at org.apache.hadoop.hive.common.FileUtils.mkdir(FileUtils.java:528)
at org.apache.hadoop.hive.ql.Context.getStagingDir(Context.java:366)
... 27 more
Caused by: java.net.UnknownHostException
... 52 more
{code}
In HADOOP core-site.xml:
{code:java}
<property>
<name>fs.default.name</name>
<value>hdfs://ns-offline</value>
</property>
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)