You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Brad Ruderman <br...@radiumone.com> on 2013/09/17 23:22:46 UTC
Export/Import Table in Hive NPE
Hi All-
I am trying to export a table in Hive 0.9, then import it into Hive 0.10
staging. Essentially moving data from a production import to staging.
I used the EXPORT table command, however when I try to import the table
back into staging I receive the following (pulled from the hive.log file).
Could anyone help to point me in what i should be looking that could cause
the problem? This is a hive managed table in the source instance, where it
was originally moved into hive by Sqoop.
Thanks,
Brad
2013-09-17 14:10:27,482 INFO parse.ParseDriver
(ParseDriver.java:parse(433)) - Parsing command: IMPORT FROM
'hdfs://user/hdfs/test_table'
2013-09-17 14:10:27,482 INFO parse.ParseDriver
(ParseDriver.java:parse(450)) - Parse Completed
2013-09-17 14:10:27,486 ERROR ql.Driver (SessionState.java:printError(427))
- FAILED: SemanticException Exception while processing
org.apache.hadoop.hive.ql.parse.SemanticException: Exception while
processing
at
org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:277)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:459)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:349)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:938)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.IllegalArgumentException:
java.net.UnknownHostException: user
at
org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:414)
at
org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:164)
at
org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:448)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:410)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:128)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2308)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:87)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2342)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2324)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:351)
at
org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:87)
... 15 more
Caused by: java.net.UnknownHostException: user
... 27 more
2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogEnd(115)) -
</PERFLOG method=compile start=1379452227481 end=1379452227487 duration=6>
2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogBegin(88))
- <PERFLOG method=releaseLocks>
2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogEnd(115)) -
</PERFLOG method=releaseLocks start=1379452227487 end=1379452227487
duration=0>
2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogBegin(88))
- <PERFLOG method=releaseLocks>
2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogEnd(115)) -
</PERFLOG method=releaseLocks start=1379452227487 end=1379452227487
duration=0>
Re: Export/Import Table in Hive NPE
Posted by Brad Ruderman <br...@radiumone.com>.
Hi All-
Just wondering if anyone had any ideas. The error message I am getting now
when exporting a managed table from hive 0.9 to hive .10 is:
13/09/18 13:14:02 ERROR ql.Driver: FAILED: SemanticException Exception
while processing
org.apache.hadoop.hive.ql.parse.SemanticException: Exception while
processing
at
org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:277)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:459)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:349)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:938)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:347)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:706)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.NullPointerException
at java.util.ArrayList.<init>(ArrayList.java:131)
at
org.apache.hadoop.hive.ql.plan.CreateTableDesc.<init>(CreateTableDesc.java:128)
at
org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:99)
... 16 more
13/09/18 13:14:02 INFO ql.Driver: </PERFLOG method=compile
start=1379535241411 end=1379535242332 duration=921>
13/09/18 13:14:02 INFO ql.Driver: <PERFLOG method=releaseLocks>
13/09/18 13:14:02 INFO ql.Driver: </PERFLOG method=releaseLocks
start=1379535242332 end=1379535242332 duration=0>
13/09/18 13:14:02 INFO ql.Driver: <PERFLOG method=releaseLocks>
13/09/18 13:14:02 INFO ql.Driver: </PERFLOG method=releaseLocks
start=1379535242333 end=1379535242333 duration=0>
Thanks,
Brad
On Tue, Sep 17, 2013 at 2:22 PM, Brad Ruderman <br...@radiumone.com>wrote:
> Hi All-
> I am trying to export a table in Hive 0.9, then import it into Hive 0.10
> staging. Essentially moving data from a production import to staging.
>
> I used the EXPORT table command, however when I try to import the table
> back into staging I receive the following (pulled from the hive.log file).
> Could anyone help to point me in what i should be looking that could cause
> the problem? This is a hive managed table in the source instance, where it
> was originally moved into hive by Sqoop.
>
> Thanks,
> Brad
>
>
>
> 2013-09-17 14:10:27,482 INFO parse.ParseDriver
> (ParseDriver.java:parse(433)) - Parsing command: IMPORT FROM
> 'hdfs://user/hdfs/test_table'
> 2013-09-17 14:10:27,482 INFO parse.ParseDriver
> (ParseDriver.java:parse(450)) - Parse Completed
> 2013-09-17 14:10:27,486 ERROR ql.Driver
> (SessionState.java:printError(427)) - FAILED: SemanticException Exception
> while processing
> org.apache.hadoop.hive.ql.parse.SemanticException: Exception while
> processing
> at
> org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:277)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:459)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:349)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:938)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.IllegalArgumentException:
> java.net.UnknownHostException: user
> at
> org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:164)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:448)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:410)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:128)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2308)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:87)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2342)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2324)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:351)
> at
> org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:87)
> ... 15 more
> Caused by: java.net.UnknownHostException: user
> ... 27 more
>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogEnd(115))
> - </PERFLOG method=compile start=1379452227481 end=1379452227487 duration=6>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogBegin(88))
> - <PERFLOG method=releaseLocks>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogEnd(115))
> - </PERFLOG method=releaseLocks start=1379452227487 end=1379452227487
> duration=0>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogBegin(88))
> - <PERFLOG method=releaseLocks>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogEnd(115))
> - </PERFLOG method=releaseLocks start=1379452227487 end=1379452227487
> duration=0>
>
Re: Export/Import Table in Hive NPE
Posted by Brad Ruderman <br...@radiumone.com>.
Hi All-
I have opened up a ticket for this issue:
https://issues.apache.org/jira/browse/HIVE-5318
Can anyone repo to confirm its a bug with Hive and not with a configuration
within my instance?
THanks,
Brad
On Tue, Sep 17, 2013 at 2:22 PM, Brad Ruderman <br...@radiumone.com>wrote:
> Hi All-
> I am trying to export a table in Hive 0.9, then import it into Hive 0.10
> staging. Essentially moving data from a production import to staging.
>
> I used the EXPORT table command, however when I try to import the table
> back into staging I receive the following (pulled from the hive.log file).
> Could anyone help to point me in what i should be looking that could cause
> the problem? This is a hive managed table in the source instance, where it
> was originally moved into hive by Sqoop.
>
> Thanks,
> Brad
>
>
>
> 2013-09-17 14:10:27,482 INFO parse.ParseDriver
> (ParseDriver.java:parse(433)) - Parsing command: IMPORT FROM
> 'hdfs://user/hdfs/test_table'
> 2013-09-17 14:10:27,482 INFO parse.ParseDriver
> (ParseDriver.java:parse(450)) - Parse Completed
> 2013-09-17 14:10:27,486 ERROR ql.Driver
> (SessionState.java:printError(427)) - FAILED: SemanticException Exception
> while processing
> org.apache.hadoop.hive.ql.parse.SemanticException: Exception while
> processing
> at
> org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:277)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:459)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:349)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:938)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.IllegalArgumentException:
> java.net.UnknownHostException: user
> at
> org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:164)
> at
> org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:129)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:448)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:410)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:128)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2308)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:87)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2342)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2324)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:351)
> at
> org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:87)
> ... 15 more
> Caused by: java.net.UnknownHostException: user
> ... 27 more
>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogEnd(115))
> - </PERFLOG method=compile start=1379452227481 end=1379452227487 duration=6>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogBegin(88))
> - <PERFLOG method=releaseLocks>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogEnd(115))
> - </PERFLOG method=releaseLocks start=1379452227487 end=1379452227487
> duration=0>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogBegin(88))
> - <PERFLOG method=releaseLocks>
> 2013-09-17 14:10:27,487 INFO ql.Driver (PerfLogger.java:PerfLogEnd(115))
> - </PERFLOG method=releaseLocks start=1379452227487 end=1379452227487
> duration=0>
>