You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hbase.apache.org by "leiwangouc@gmail.com" <le...@gmail.com> on 2014/08/15 12:01:21 UTC

Connection refused when connecting to hbase from mapper task

In the following code,  I just want to connect to hbase and do a get operation in a mapper.
Any insight on this?

Thanks,
Lei

public class TagMapper extends Mapper<LongWritable, Text, Text, Text> { 

private HTable hTable; 

@Override 
protected void setup(Context context){ 
try { 
    hTable = new HTable(context.getConfiguration(), "userdigest"); 
} catch (IOException e) { 
    e.printStackTrace(); 
} 
} 

@Override 
public void map(LongWritable key, Text value, Context context) 
throws IOException, InterruptedException { 

String agid = value.toString(); 
Get get = new Get(Bytes.toBytes(agid)); 
Result rs = hTable.get(get); 
for(KeyValue kv:rs.raw()){ 

context.write(new Text(new String(kv.getRow())+":" + new String(kv.getQualifier())), new Text(new String(kv.getValue()))); 

} 
} 
}

The task error log: 
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2014-08-15 17:43:11,102 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2014-08-15 17:43:11,103 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration)
2014-08-15 17:43:11,103 ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
2014-08-15 17:43:11,103 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
	at org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
	at org.apache.hadoop.hbase.zookeeper.ClusterId.getId(ClusterId.java:50)
	at org.apache.hadoop.hbase.zookeeper.ClusterId.hasId(ClusterId.java:44)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:615)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:871)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:846)
	at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:271)
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:211)
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:170)
	at com.agrantsem.data.mapreduce.userdigest.TagMapper.setup(TagMapper.java:23)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:138)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
2014-08-15 17:43:11,103 WARN org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect


leiwangouc@gmail.com

Re: Re: Connection refused when connecting to hbase from mapper task

Posted by "leiwangouc@gmail.com" <le...@gmail.com>.

Thanks,  this is the problem. 


leiwangouc@gmail.com
 
From: Ted Yu
Date: 2014-08-15 18:27
To: user@hbase.apache.org
CC: user
Subject: Re: Connection refused when connecting to hbase from mapper task
Is hbase-site.xml in the classpath of map task ?
 
Cheers
 
On Aug 15, 2014, at 3:01 AM, "leiwangouc@gmail.com" <le...@gmail.com> wrote:
 
> 
> In the following code,  I just want to connect to hbase and do a get operation in a mapper.
> Any insight on this?
> 
> Thanks,
> Lei
> 
> public class TagMapper extends Mapper<LongWritable, Text, Text, Text> { 
> 
> private HTable hTable; 
> 
> @Override 
> protected void setup(Context context){ 
> try { 
>    hTable = new HTable(context.getConfiguration(), "userdigest"); 
> } catch (IOException e) { 
>    e.printStackTrace(); 
> } 
> } 
> 
> @Override 
> public void map(LongWritable key, Text value, Context context) 
> throws IOException, InterruptedException { 
> 
> String agid = value.toString(); 
> Get get = new Get(Bytes.toBytes(agid)); 
> Result rs = hTable.get(get); 
> for(KeyValue kv:rs.raw()){ 
> 
> context.write(new Text(new String(kv.getRow())+":" + new String(kv.getQualifier())), new Text(new String(kv.getValue()))); 
> 
> } 
> } 
> }
> 
> The task error log: 
> java.net.ConnectException: Connection refused
>    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
>    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
> 2014-08-15 17:43:11,102 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
> 2014-08-15 17:43:11,103 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration)
> 2014-08-15 17:43:11,103 ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
> 2014-08-15 17:43:11,103 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection Unable to set watcher on znode (/hbase/hbaseid)
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
>    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>    at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
>    at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
>    at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
>    at org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
>    at org.apache.hadoop.hbase.zookeeper.ClusterId.getId(ClusterId.java:50)
>    at org.apache.hadoop.hbase.zookeeper.ClusterId.hasId(ClusterId.java:44)
>    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:615)
>    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:871)
>    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:846)
>    at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:271)
>    at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:211)
>    at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:170)
>    at com.agrantsem.data.mapreduce.userdigest.TagMapper.setup(TagMapper.java:23)
>    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:138)
>    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
>    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>    at java.security.AccessController.doPrivileged(Native Method)
>    at javax.security.auth.Subject.doAs(Subject.java:396)
>    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>    at org.apache.hadoop.mapred.Child.main(Child.java:262)
> 2014-08-15 17:43:11,103 WARN org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
> 
> 
> leiwangouc@gmail.com

Re: Connection refused when connecting to hbase from mapper task

Posted by Ted Yu <yu...@gmail.com>.

Is hbase-site.xml in the classpath of map task ?

Cheers

On Aug 15, 2014, at 3:01 AM, "leiwangouc@gmail.com" <le...@gmail.com> wrote:

> 
> In the following code,  I just want to connect to hbase and do a get operation in a mapper.
> Any insight on this?
> 
> Thanks,
> Lei
> 
> public class TagMapper extends Mapper<LongWritable, Text, Text, Text> { 
> 
> private HTable hTable; 
> 
> @Override 
> protected void setup(Context context){ 
> try { 
>    hTable = new HTable(context.getConfiguration(), "userdigest"); 
> } catch (IOException e) { 
>    e.printStackTrace(); 
> } 
> } 
> 
> @Override 
> public void map(LongWritable key, Text value, Context context) 
> throws IOException, InterruptedException { 
> 
> String agid = value.toString(); 
> Get get = new Get(Bytes.toBytes(agid)); 
> Result rs = hTable.get(get); 
> for(KeyValue kv:rs.raw()){ 
> 
> context.write(new Text(new String(kv.getRow())+":" + new String(kv.getQualifier())), new Text(new String(kv.getValue()))); 
> 
> } 
> } 
> }
> 
> The task error log: 
> java.net.ConnectException: Connection refused
>    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
>    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
>    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
> 2014-08-15 17:43:11,102 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
> 2014-08-15 17:43:11,103 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration)
> 2014-08-15 17:43:11,103 ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
> 2014-08-15 17:43:11,103 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection Unable to set watcher on znode (/hbase/hbaseid)
> org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
>    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
>    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>    at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
>    at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
>    at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
>    at org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
>    at org.apache.hadoop.hbase.zookeeper.ClusterId.getId(ClusterId.java:50)
>    at org.apache.hadoop.hbase.zookeeper.ClusterId.hasId(ClusterId.java:44)
>    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:615)
>    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:871)
>    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:846)
>    at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:271)
>    at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:211)
>    at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:170)
>    at com.agrantsem.data.mapreduce.userdigest.TagMapper.setup(TagMapper.java:23)
>    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:138)
>    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
>    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
>    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
>    at java.security.AccessController.doPrivileged(Native Method)
>    at javax.security.auth.Subject.doAs(Subject.java:396)
>    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>    at org.apache.hadoop.mapred.Child.main(Child.java:262)
> 2014-08-15 17:43:11,103 WARN org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
> 
> 
> leiwangouc@gmail.com