You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Jacob Perkins <ja...@gmail.com> on 2011/01/27 23:58:33 UTC

HBaseStorage fails to load data

Running this script:
        
data = LOAD '$TABLE' USING HBaseStorage('$CF:field_1');
DUMP data;

fails with the following error:

Failed Jobs:
JobId	Alias	Feature	Message	Outputs
N/A	data	MAP_ONLY	Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Call to /10.117.86.81:60020 failed on local exception: java.io.EOFException
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:280)
	at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:899)
	at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:916)
	at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:834)
	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:767)
	at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
	at org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
	at org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Call to /10.117.86.81:60020 failed on local exception: java.io.EOFException
	at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:788)
	at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:757)
	at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
	at $Proxy6.getProtocolVersion(Unknown Source)
	at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
	at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
	at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
	at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:952)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:969)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:685)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:584)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:559)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:782)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:588)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:553)
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
	at com.infochimps.hbase.pig.HBaseStorage.setLocation(HBaseStorage.java:279)
	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:261)
	... 14 more
Caused by: java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:375)
	at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:521)
	at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:459)
	hdfs://ip-10-113-50-207.ec2.internal/tmp/temp-1162684483/tmp278494015



I'm not sure what resource, exactly, it's looking for and not finding. Any ideas? Thanks,

--jacob
@thedatachef

Re: HBaseStorage fails to load data

Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Looks like your zookeeper went away.
You should ping the hbase user list... getting close to out of my depth for
email troubleshooting :)

On Thu, Jan 27, 2011 at 3:24 PM, Jacob Perkins <ja...@gmail.com>wrote:

> So here's what I can see in the thrift logs:
>
> java.net.ConnectException: Connection refused
>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>        at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>        at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)
> 2011-01-25 23:05:16,399 INFO org.apache.zookeeper.ClientCnxn: Opening
> socket connection to server ip-10-113-50-207.ec2.internal/
> 10.113.50.207:2181
> 2011-01-25 23:05:16,410 WARN org.apache.zookeeper.ClientCnxn: Session 0x0
> for server null, unexpected error, closing socket connection and attempting
> reconnect
> java.net.ConnectException: Connection refused
>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>        at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>        at
> org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)
> 2011-01-25 23:05:16,510 INFO
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: getMaster
> attempt 9 of 10 failed; no more retrying.
> java.io.IOException:
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
>        at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKeeperWrapper.java:481)
>        at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readMasterAddressOrThrow(ZooKeeperWrapper.java:377)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:381)
>        at
> org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:78)
>        at
> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:191)
>        at
> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:817)
>        at
> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:874)
> Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /hbase/master
>        at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
>        at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
>        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:921)
>        at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKeeperWrapper.java:477)
>        ... 6 more
>
> Even more confusing...
>
> --jacob
> @thedatachef
>
>
> On Jan 27, 2011, at 5:19 PM, Dmitriy Ryaboy wrote:
>
> > Looks like it's trying to create the HTable object (to get the splits)
> and
> > failing.
> > Check the hbase logs.. it's making the call to hbase ok, and failing
> there
> >
> > On Thu, Jan 27, 2011 at 2:58 PM, Jacob Perkins <
> jacob.a.perkins@gmail.com>wrote:
> >
> >> Running this script:
> >>
> >> data = LOAD '$TABLE' USING HBaseStorage('$CF:field_1');
> >> DUMP data;
> >>
> >> fails with the following error:
> >>
> >> Failed Jobs:
> >> JobId   Alias   Feature Message Outputs
> >> N/A     data    MAP_ONLY        Message:
> >> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Call
> to /
> >> 10.117.86.81:60020 failed on local exception: java.io.EOFException
> >>       at
> >>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:280)
> >>       at
> >> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:899)
> >>       at
> >> org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:916)
> >>       at
> org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
> >>       at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:834)
> >>       at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
> >>       at java.security.AccessController.doPrivileged(Native Method)
> >>       at javax.security.auth.Subject.doAs(Subject.java:396)
> >>       at
> >>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
> >>       at
> >> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
> >>       at
> org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:767)
> >>       at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
> >>       at
> >>
> org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
> >>       at
> >> org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
> >>       at java.lang.Thread.run(Thread.java:662)
> >> Caused by: java.io.IOException: Call to /10.117.86.81:60020 failed on
> >> local exception: java.io.EOFException
> >>       at
> >>
> org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:788)
> >>       at
> >> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:757)
> >>       at
> >> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
> >>       at $Proxy6.getProtocolVersion(Unknown Source)
> >>       at
> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
> >>       at
> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
> >>       at
> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
> >>       at
> >> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)
> >>       at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:952)
> >>       at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:969)
> >>       at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:685)
> >>       at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:584)
> >>       at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:559)
> >>       at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:782)
> >>       at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:588)
> >>       at
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:553)
> >>       at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
> >>       at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
> >>       at
> >> com.infochimps.hbase.pig.HBaseStorage.setLocation(HBaseStorage.java:279)
> >>       at
> >>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:261)
> >>       ... 14 more
> >> Caused by: java.io.EOFException
> >>       at java.io.DataInputStream.readInt(DataInputStream.java:375)
> >>       at
> >>
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:521)
> >>       at
> >>
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:459)
> >>
> >> hdfs://ip-10-113-50-207.ec2.internal/tmp/temp-1162684483/tmp278494015
> >>
> >>
> >>
> >> I'm not sure what resource, exactly, it's looking for and not finding.
> Any
> >> ideas? Thanks,
> >>
> >> --jacob
> >> @thedatachef
>
>

Re: HBaseStorage fails to load data

Posted by Jacob Perkins <ja...@gmail.com>.
So here's what I can see in the thrift logs:

java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)
2011-01-25 23:05:16,399 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server ip-10-113-50-207.ec2.internal/10.113.50.207:2181
2011-01-25 23:05:16,410 WARN org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)
2011-01-25 23:05:16,510 INFO org.apache.hadoop.hbase.client.HConnectionManager$TableServers: getMaster attempt 9 of 10 failed; no more retrying.
java.io.IOException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKeeperWrapper.java:481)
	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readMasterAddressOrThrow(ZooKeeperWrapper.java:377)
	at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:381)
	at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:78)
	at org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:191)
	at org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:817)
	at org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:874)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:921)
	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKeeperWrapper.java:477)
	... 6 more

Even more confusing...

--jacob
@thedatachef


On Jan 27, 2011, at 5:19 PM, Dmitriy Ryaboy wrote:

> Looks like it's trying to create the HTable object (to get the splits) and
> failing.
> Check the hbase logs.. it's making the call to hbase ok, and failing there
> 
> On Thu, Jan 27, 2011 at 2:58 PM, Jacob Perkins <ja...@gmail.com>wrote:
> 
>> Running this script:
>> 
>> data = LOAD '$TABLE' USING HBaseStorage('$CF:field_1');
>> DUMP data;
>> 
>> fails with the following error:
>> 
>> Failed Jobs:
>> JobId   Alias   Feature Message Outputs
>> N/A     data    MAP_ONLY        Message:
>> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Call to /
>> 10.117.86.81:60020 failed on local exception: java.io.EOFException
>>       at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:280)
>>       at
>> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:899)
>>       at
>> org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:916)
>>       at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
>>       at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:834)
>>       at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
>>       at java.security.AccessController.doPrivileged(Native Method)
>>       at javax.security.auth.Subject.doAs(Subject.java:396)
>>       at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>>       at
>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
>>       at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:767)
>>       at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
>>       at
>> org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
>>       at
>> org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
>>       at java.lang.Thread.run(Thread.java:662)
>> Caused by: java.io.IOException: Call to /10.117.86.81:60020 failed on
>> local exception: java.io.EOFException
>>       at
>> org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:788)
>>       at
>> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:757)
>>       at
>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>       at $Proxy6.getProtocolVersion(Unknown Source)
>>       at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>       at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>       at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>       at
>> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)
>>       at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:952)
>>       at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:969)
>>       at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:685)
>>       at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:584)
>>       at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:559)
>>       at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:782)
>>       at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:588)
>>       at
>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:553)
>>       at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
>>       at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
>>       at
>> com.infochimps.hbase.pig.HBaseStorage.setLocation(HBaseStorage.java:279)
>>       at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:261)
>>       ... 14 more
>> Caused by: java.io.EOFException
>>       at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>       at
>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:521)
>>       at
>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:459)
>> 
>> hdfs://ip-10-113-50-207.ec2.internal/tmp/temp-1162684483/tmp278494015
>> 
>> 
>> 
>> I'm not sure what resource, exactly, it's looking for and not finding. Any
>> ideas? Thanks,
>> 
>> --jacob
>> @thedatachef


Re: HBaseStorage fails to load data

Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Looks like it's trying to create the HTable object (to get the splits) and
failing.
Check the hbase logs.. it's making the call to hbase ok, and failing there

On Thu, Jan 27, 2011 at 2:58 PM, Jacob Perkins <ja...@gmail.com>wrote:

> Running this script:
>
> data = LOAD '$TABLE' USING HBaseStorage('$CF:field_1');
> DUMP data;
>
> fails with the following error:
>
> Failed Jobs:
> JobId   Alias   Feature Message Outputs
> N/A     data    MAP_ONLY        Message:
> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Call to /
> 10.117.86.81:60020 failed on local exception: java.io.EOFException
>        at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:280)
>        at
> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:899)
>        at
> org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:916)
>        at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
>        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:834)
>        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:793)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at javax.security.auth.Subject.doAs(Subject.java:396)
>        at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>        at
> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:793)
>        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:767)
>        at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)
>        at
> org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)
>        at
> org.apache.hadoop.mapred.jobcontrol.JobControl.run(JobControl.java:279)
>        at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Call to /10.117.86.81:60020 failed on
> local exception: java.io.EOFException
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:788)
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:757)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>        at $Proxy6.getProtocolVersion(Unknown Source)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:952)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:969)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:685)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:584)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:559)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:782)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:588)
>        at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:553)
>        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:171)
>        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:145)
>        at
> com.infochimps.hbase.pig.HBaseStorage.setLocation(HBaseStorage.java:279)
>        at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:261)
>        ... 14 more
> Caused by: java.io.EOFException
>        at java.io.DataInputStream.readInt(DataInputStream.java:375)
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:521)
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:459)
>
>  hdfs://ip-10-113-50-207.ec2.internal/tmp/temp-1162684483/tmp278494015
>
>
>
> I'm not sure what resource, exactly, it's looking for and not finding. Any
> ideas? Thanks,
>
> --jacob
> @thedatachef