You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by fengcheng he <he...@gmail.com> on 2011/04/15 11:32:39 UTC
pig mismatch hadoop version problem
Hi friends,
My problem is that:
The Pig version is 0.8.0 ,but for some reason I have to use the hadoop
of 0.19.x.
And the log information is as following:
Error before Pig is launched
----------------------------
ERROR 2999: Unexpected internal error. Failed to create DataStorage
java.lang.RuntimeException: Failed to create DataStorage
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:213)
at
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:133)
at org.apache.pig.impl.PigContext.connect(PigContext.java:183)
at org.apache.pig.PigServer.<init>(PigServer.java:225)
at org.apache.pig.PigServer.<init>(PigServer.java:214)
at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55)
at org.apache.pig.Main.run(Main.java:462)
at org.apache.pig.Main.main(Main.java:107)
Caused by: java.io.IOException: Call to xxxx.xxxx.xxxx <http://not the same
as the log ,but I changes it for some reason>.com/10.36.98.23:54310 failed
on local exception: java.io.EOFException
at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
at org.apache.hadoop.ipc.Client.call(Client.java:743)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at $Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
at
org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72)
... 9 more
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:375)
at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
================================================================================
And I serch the similar problem
http://mail-archives.apache.org/mod_mbox/hadoop-pig-user/200904.mbox/%3c004f01c9c6f2$b1222500$13666f00$@com%3e
But I do not know what to deal with it in detail?
Could you help me?
Thanks.
Re: pig mismatch hadoop version problem
Posted by Alan Gates <ga...@yahoo-inc.com>.
Pig 0.8 only works with Hadoop 0.20.2. There is no version of Pig
that works with 0.19 out of the box. Pig 0.4, patched as explained
here https://issues.apache.org/jira/browse/PIG-573 can be made to work
with 0.19.
Alan.
On Apr 15, 2011, at 2:32 AM, fengcheng he wrote:
> Hi friends,
>
> My problem is that:
> The Pig version is 0.8.0 ,but for some reason I have to use the
> hadoop
> of 0.19.x.
> And the log information is as following:
> Error before Pig is launched
> ----------------------------
> ERROR 2999: Unexpected internal error. Failed to create DataStorage
>
> java.lang.RuntimeException: Failed to create DataStorage
> at
> org
> .apache
> .pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:
> 75)
> at
> org
> .apache
> .pig
> .backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
> at
> org
> .apache
> .pig
> .backend
> .hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:
> 213)
> at
> org
> .apache
> .pig
> .backend
> .hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:
> 133)
> at org.apache.pig.impl.PigContext.connect(PigContext.java:183)
> at org.apache.pig.PigServer.<init>(PigServer.java:225)
> at org.apache.pig.PigServer.<init>(PigServer.java:214)
> at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55)
> at org.apache.pig.Main.run(Main.java:462)
> at org.apache.pig.Main.main(Main.java:107)
> Caused by: java.io.IOException: Call to xxxx.xxxx.xxxx <http://not
> the same
> as the log ,but I changes it for some reason>.com/10.36.98.23:54310
> failed
> on local exception: java.io.EOFException
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
> at org.apache.hadoop.ipc.Client.call(Client.java:743)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
> at $Proxy0.getProtocolVersion(Unknown Source)
> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
> at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:
> 106)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207)
> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170)
> at
> org
> .apache
> .hadoop
> .hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:
> 1378)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95)
> at
> org
> .apache
> .pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:
> 72)
> ... 9 more
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.ipc.Client
> $Connection.receiveResponse(Client.java:501)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446)
> =
> =
> =
> =
> =
> =
> =
> =
> =
> =
> ======================================================================
> And I serch the similar problem
> http://mail-archives.apache.org/mod_mbox/hadoop-pig-user/200904.mbox/%3c004f01c9c6f2$b1222500$13666f00$@com%3e
> But I do not know what to deal with it in detail?
> Could you help me?
> Thanks.