You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Puneet Khatod <pu...@tavant.com> on 2012/05/31 10:35:09 UTC
Error in Hive execution on cluster : Wrong FS: hdfs
Hi,
I am facing below error when I am firing any query in Hive.
My Hive setup is present on the master node in my cluster. Hadoop is configured using IP_addresses in configuration xmls and in master, slave files and it is running fine. The error only arises when hive query is executed which had table location on HDFS. It seems hive is expecting configurations to be done using hostnames.
Please help me in configuring hive such that it can understand IP address based configuration. I am using Hadoop 0.20.2 and Hive 0.7.1.
Regards,
Puneet
____________________ ______________________ ______________________________ ______________________________
hive> show tables;
OK
test_page_view
test_page_view_stg
Time taken: 131.309 seconds
hive>
> select * from test_page_view_stg;
FAILED: Hive Internal Error: java.lang.RuntimeException(Error while making MR scratch directory - check filesystem config (null))
java.lang.RuntimeException: Error while making MR scratch directory - check filesystem config (null)
at org.apache.hadoop.hive.ql.Context.getMRScratchDir(Context.java:196)
at org.apache.hadoop.hive.ql.Context.getMRTmpFileURI(Context.java:247)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:900)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6594)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.IllegalArgumentException: Wrong FS: hdfs://<IP_ADDRESS>:9100/tmp/hive-hadoop/hive_2012-05-31_11-29-19_844_3368974040204630542, expected: hdfs://<hostname>.local:9100
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
at org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:99)
at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:222)
at org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:116)
at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:146)
at org.apache.hadoop.hive.ql.Context.getMRScratchDir(Context.java:190)
... 14 more
Any comments or statements made in this email are not necessarily those of Tavant Technologies.
The information transmitted is intended only for the person or entity to which it is addressed and may
contain confidential and/or privileged material. If you have received this in error, please contact the
sender and delete the material from any computer. All e-mails sent from or to Tavant Technologies
may be subject to our monitoring procedures.
Re: Error in Hive execution on cluster : Wrong FS: hdfs
Posted by Abhishek Pratap Singh <ma...@gmail.com>.
Can you check the configuration in your Hadoop's core-site.xml file.
Also are you able to fire fs command from hadoop successfully by using the
fully qualified directory structure?
Seems a problem in your file system config.
Regards,
Abhishek
On Thu, May 31, 2012 at 1:35 AM, Puneet Khatod <pu...@tavant.com>wrote:
> Hi,****
>
> ** **
>
> I am facing below error when I am firing any query in Hive.****
>
> My Hive setup is present on the master node in my cluster. Hadoop is
> configured using IP_addresses in configuration xmls and in master, slave
> files and it is running fine. The error only arises when hive query is
> executed which had table location on HDFS. It seems hive is expecting
> configurations to be done using hostnames.****
>
> ** **
>
> Please help me in configuring hive such that it can understand IP address
> based configuration. I am using Hadoop 0.20.2 and Hive 0.7.1.****
>
> ** **
>
> Regards,****
>
> Puneet****
>
> ** **
>
> ____________________ ______________________
> ______________________________ ______________________________****
>
> ** **
>
> hive> show tables;****
>
> OK****
>
> test_page_view****
>
> test_page_view_stg****
>
> Time taken: 131.309 seconds****
>
> hive>****
>
> > select * from test_page_view_stg;****
>
> FAILED: Hive Internal Error: java.lang.RuntimeException(Error while making
> MR scratch directory - check filesystem config (null))****
>
> java.lang.RuntimeException: Error while making MR scratch directory -
> check filesystem config (null)****
>
> at
> org.apache.hadoop.hive.ql.Context.getMRScratchDir(Context.java:196)****
>
> at
> org.apache.hadoop.hive.ql.Context.getMRTmpFileURI(Context.java:247)****
>
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:900)
> ****
>
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6594)
> ****
>
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> ****
>
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340)****
>
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736)****
>
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)****
>
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)****
>
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456)**
> **
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)****
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ****
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ****
>
> at java.lang.reflect.Method.invoke(Method.java:601)****
>
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)****
>
> *Caused by: java.lang.IllegalArgumentException: Wrong FS:
> hdfs://<IP_ADDRESS>:9100/tmp/hive-hadoop/hive_2012-05-31_11-29-19_844_3368974040204630542,
> expected: hdfs://<hostname>.local:9100*
>
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)*
> ***
>
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:99)
> ****
>
> at
> org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:222)****
>
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:116)
> ****
>
> at
> org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:146)****
>
> at
> org.apache.hadoop.hive.ql.Context.getMRScratchDir(Context.java:190)****
>
> ... 14 more****
>
> ** **
>
> Any comments or statements made in this email are not necessarily those of
> Tavant Technologies.
> The information transmitted is intended only for the person or entity to
> which it is addressed and may
> contain confidential and/or privileged material. If you have received this
> in error, please contact the
> sender and delete the material from any computer. All e-mails sent from or
> to Tavant Technologies
> may be subject to our monitoring procedures.
>