You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by al...@aim.com on 2012/08/20 00:10:53 UTC

fetcher fails on connection error in nutch-2.0 with hbase

After fetching for about 18 hours fetcher throws this error

java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
        at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
        at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
        at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
        at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
        at $Proxy6.getClosestRowBefore(Unknown Source)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:947)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1524)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
        at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
        at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
        at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)


 WARN  zookeeper.ClientCnxn - Session 0x1393cf29d5e0003 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
2012-08-19 13:26:56,935 WARN  zookeeper.ClientCnxn - Session 0x1393cf29d5e0003 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
2012-08-19 13:26:57,075 WARN  zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase

 
repeadetely and then fails. I checked the hbase process was alive.

Any ideas what could cause this issue?

Thanks.
Alex.

Re: fetcher fails on connection error in nutch-2.0 with hbase

Posted by Lewis John Mcgibbney <le...@gmail.com>.
I think I've seen similar traces when running gora-hbase tests.
In my case it seemed to be some proxy problems (I think relating to
tinyproxy) which just mucked everything up. One I found a workstation
which didn't have tiny proxy installed the messages vanished and I was
able to continue.

I am sorry I can't be of more help further than this.

I really don't think this is Nutch related but again I can't 100% confirm.

Lewis

On Sun, Aug 19, 2012 at 11:10 PM,  <al...@aim.com> wrote:
> After fetching for about 18 hours fetcher throws this error
>
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
>         at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>         at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
>         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
>         at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
>         at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1045)
>         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
>         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
>         at $Proxy6.getClosestRowBefore(Unknown Source)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:947)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:814)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:788)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1024)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:818)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1524)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1409)
>         at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:943)
>         at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
>         at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
>
>
>  WARN  zookeeper.ClientCnxn - Session 0x1393cf29d5e0003 for server null, unexpected error, closing socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
>         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
> 2012-08-19 13:26:56,935 WARN  zookeeper.ClientCnxn - Session 0x1393cf29d5e0003 for server null, unexpected error, closing socket connection and attempting reconnect
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>         at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
>         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
> 2012-08-19 13:26:57,075 WARN  zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
>
>
> repeadetely and then fails. I checked the hbase process was alive.
>
> Any ideas what could cause this issue?
>
> Thanks.
> Alex.



-- 
Lewis