You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by "Lanati, Matteo" <Ma...@lrz.de> on 2013/08/29 10:33:04 UTC

HBase client with security

Hi all,

I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
Regarding HBase client configuration:

- hbase-site.xml specifies

  <property>
    <name>hbase.security.authentication</name>
    <value>kerberos</value>
  </property>
  <property>
    <name>hbase.rpc.engine</name>
    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>master.hadoop.local,host49.hadoop.local</value>
  </property>

where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"

- I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.

Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log

2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
org.apache.hadoop.security.AccessControlException: Authentication is required
	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
Thanks,

Matteo


Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724

RE: HBase client with security

Posted by "Lanati, Matteo" <Ma...@lrz.de>.
Hi again Harsh,

I had a closer look at the HBase configuration file and I found out that the hbase.master.kerberos.principal was missing.
Once I add it, the client started to work.
I was deceived by the fact that in the error message there was a privilege exception associated to my user, so I investigated into that. I only noticed later the sentence 'Failed to specify server's Kerberos principal name' and the fact that it didn't refer to the KDC but to the HBase master.
The issue can be closed. I hope this can help someone else too.
All the best,

Matteo


Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748 Garching b. München (Germany)
Phone: +49 89 35831 8724

________________________________________
From: Lanati, Matteo [Matteo.Lanati@lrz.de]
Sent: 29 August 2013 23:54
To: user@hbase.apache.org
Subject: RE: HBase client with security

Hi Harsh,

thanks for the suggestion.
I added HADOOP_PREFIX so that the conf folder is in the path.
It still doesn't work, so I suppose Hadoop's core-site.xml is faulty (though I need a Kerberos ticket to use Hadoop, so security is working).
In fact, when I try to list from HBase shell I get

13/08/29 23:47:43 ERROR security.UserGroupInformation: PriviledgedActionException as:lu95jib@HADOOP.LRZ.DE cause:java.io.IOException: Failed to specify server's Kerberos principal name
13/08/29 23:47:43 INFO security.UserGroupInformation: Initiating logout for lu95jib@HADOOP.LRZ.DE
13/08/29 23:47:43 INFO security.UserGroupInformation: Initiating re-login for lu95jib@HADOOP.LRZ.DE


The file core-site.xml contains the following

    <name>fs.default.name</name>
    <value>hdfs://10.156.120.41:9000</value>
  </property>

  <property>
    <name>hadoop.security.authentication</name>
    <value>kerberos</value>
  </property>

  <property>
    <name>hadoop.security.authorization</name>
    <value>true</value>
  </property>

  <property>
    <name>hadoop.kerberos.kinit.command</name>
    <value>/usr/bin/kinit</value>
  </property>

What else should I need? Maybe a reference to the keytab contained in  hbase/conf/zk-jaas.conf?

Bye,

Matteo


Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748 Garching b. München (Germany)
Phone: +49 89 35831 8724

________________________________________
From: Harsh J [harsh@cloudera.com]
Sent: 29 August 2013 15:53
To: user@hbase.apache.org
Subject: Re: HBase client with security

Two things come to mind:

1. Is HADOOP_CONF_DIR also on HBase's classpath? If it or
HADOOP_PREFIX/HADOOP_HOME is defined, it usually is. But re-check via
"hbase classpath"
2. Assuming (1) is good, does your core-site.xml have kerberos
authentication settings for hadoop as well?

On Thu, Aug 29, 2013 at 6:58 PM, Lanati, Matteo <Ma...@lrz.de> wrote:
> Hi all,
>
> I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
> HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
> I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
> Regarding HBase client configuration:
>
> - hbase-site.xml specifies
>
>  <property>
>    <name>hbase.security.authentication</name>
>    <value>kerberos</value>
>  </property>
>  <property>
>    <name>hbase.rpc.engine</name>
>    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
>  </property>
>  <property>
>    <name>hbase.zookeeper.quorum</name>
>    <value>master.hadoop.local,host49.hadoop.local</value>
>  </property>
>
> where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"
>
> - I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.
>
> Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log
>
> 2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
> org.apache.hadoop.security.AccessControlException: Authentication is required
>         at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
>
> It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
> What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
> Thanks,
>
> Matteo
>
>
> Matteo Lanati
> Distributed Resources Group
> Leibniz-Rechenzentrum (LRZ)
> Boltzmannstrasse 1
> 85748   Garching b. München     (Germany)
> Phone: +49 89 35831 8724
>
>



--
Harsh J

RE: HBase client with security

Posted by "Lanati, Matteo" <Ma...@lrz.de>.
Hi Harsh,

thanks for the suggestion.
I added HADOOP_PREFIX so that the conf folder is in the path.
It still doesn't work, so I suppose Hadoop's core-site.xml is faulty (though I need a Kerberos ticket to use Hadoop, so security is working).
In fact, when I try to list from HBase shell I get

13/08/29 23:47:43 ERROR security.UserGroupInformation: PriviledgedActionException as:lu95jib@HADOOP.LRZ.DE cause:java.io.IOException: Failed to specify server's Kerberos principal name
13/08/29 23:47:43 INFO security.UserGroupInformation: Initiating logout for lu95jib@HADOOP.LRZ.DE
13/08/29 23:47:43 INFO security.UserGroupInformation: Initiating re-login for lu95jib@HADOOP.LRZ.DE


The file core-site.xml contains the following

    <name>fs.default.name</name>
    <value>hdfs://10.156.120.41:9000</value>
  </property>

  <property>
    <name>hadoop.security.authentication</name>
    <value>kerberos</value>
  </property>

  <property>
    <name>hadoop.security.authorization</name>
    <value>true</value>
  </property>

  <property>
    <name>hadoop.kerberos.kinit.command</name>
    <value>/usr/bin/kinit</value>
  </property>

What else should I need? Maybe a reference to the keytab contained in  hbase/conf/zk-jaas.conf?

Bye,

Matteo


Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748 Garching b. München (Germany)
Phone: +49 89 35831 8724

________________________________________
From: Harsh J [harsh@cloudera.com]
Sent: 29 August 2013 15:53
To: user@hbase.apache.org
Subject: Re: HBase client with security

Two things come to mind:

1. Is HADOOP_CONF_DIR also on HBase's classpath? If it or
HADOOP_PREFIX/HADOOP_HOME is defined, it usually is. But re-check via
"hbase classpath"
2. Assuming (1) is good, does your core-site.xml have kerberos
authentication settings for hadoop as well?

On Thu, Aug 29, 2013 at 6:58 PM, Lanati, Matteo <Ma...@lrz.de> wrote:
> Hi all,
>
> I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
> HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
> I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
> Regarding HBase client configuration:
>
> - hbase-site.xml specifies
>
>  <property>
>    <name>hbase.security.authentication</name>
>    <value>kerberos</value>
>  </property>
>  <property>
>    <name>hbase.rpc.engine</name>
>    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
>  </property>
>  <property>
>    <name>hbase.zookeeper.quorum</name>
>    <value>master.hadoop.local,host49.hadoop.local</value>
>  </property>
>
> where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"
>
> - I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.
>
> Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log
>
> 2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
> org.apache.hadoop.security.AccessControlException: Authentication is required
>         at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
>
> It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
> What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
> Thanks,
>
> Matteo
>
>
> Matteo Lanati
> Distributed Resources Group
> Leibniz-Rechenzentrum (LRZ)
> Boltzmannstrasse 1
> 85748   Garching b. München     (Germany)
> Phone: +49 89 35831 8724
>
>



--
Harsh J

Re: HBase client with security

Posted by Harsh J <ha...@cloudera.com>.
Two things come to mind:

1. Is HADOOP_CONF_DIR also on HBase's classpath? If it or
HADOOP_PREFIX/HADOOP_HOME is defined, it usually is. But re-check via
"hbase classpath"
2. Assuming (1) is good, does your core-site.xml have kerberos
authentication settings for hadoop as well?

On Thu, Aug 29, 2013 at 6:58 PM, Lanati, Matteo <Ma...@lrz.de> wrote:
> Hi all,
>
> I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
> HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
> I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
> Regarding HBase client configuration:
>
> - hbase-site.xml specifies
>
>  <property>
>    <name>hbase.security.authentication</name>
>    <value>kerberos</value>
>  </property>
>  <property>
>    <name>hbase.rpc.engine</name>
>    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
>  </property>
>  <property>
>    <name>hbase.zookeeper.quorum</name>
>    <value>master.hadoop.local,host49.hadoop.local</value>
>  </property>
>
> where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"
>
> - I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.
>
> Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log
>
> 2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
> org.apache.hadoop.security.AccessControlException: Authentication is required
>         at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
>
> It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
> What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
> Thanks,
>
> Matteo
>
>
> Matteo Lanati
> Distributed Resources Group
> Leibniz-Rechenzentrum (LRZ)
> Boltzmannstrasse 1
> 85748   Garching b. München     (Germany)
> Phone: +49 89 35831 8724
>
>



-- 
Harsh J

HBase client with security

Posted by "Lanati, Matteo" <Ma...@lrz.de>.
Hi all,

I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
Regarding HBase client configuration:

- hbase-site.xml specifies

 <property>
   <name>hbase.security.authentication</name>
   <value>kerberos</value>
 </property>
 <property>
   <name>hbase.rpc.engine</name>
   <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
 </property>
 <property>
   <name>hbase.zookeeper.quorum</name>
   <value>master.hadoop.local,host49.hadoop.local</value>
 </property>

where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"

- I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.

Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log

2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
org.apache.hadoop.security.AccessControlException: Authentication is required
	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
Thanks,

Matteo


Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724

Re: HBase client with security

Posted by "Lanati, Matteo" <Ma...@lrz.de>.
Hi Devaraj,

you're right, I just subscribed, sorry for the spam.

Matteo


On Aug 29, 2013, at 3:31 PM, Devaraj k <de...@huawei.com> wrote:

> Please ask this question in user@hbase.apache.org, you would get better response there. 
> 
> Thanks
> Devaraj k
> 
> 
> -----Original Message-----
> From: Lanati, Matteo [mailto:Matteo.Lanati@lrz.de] 
> Sent: 29 August 2013 14:03
> To: <us...@hadoop.apache.org>
> Subject: HBase client with security
> 
> Hi all,
> 
> I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
> HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
> I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
> Regarding HBase client configuration:
> 
> - hbase-site.xml specifies
> 
>  <property>
>    <name>hbase.security.authentication</name>
>    <value>kerberos</value>
>  </property>
>  <property>
>    <name>hbase.rpc.engine</name>
>    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
>  </property>
>  <property>
>    <name>hbase.zookeeper.quorum</name>
>    <value>master.hadoop.local,host49.hadoop.local</value>
>  </property>
> 
> where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"
> 
> - I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.
> 
> Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log
> 
> 2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
> org.apache.hadoop.security.AccessControlException: Authentication is required
> 	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> 
> It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
> What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
> Thanks,
> 
> Matteo
> 
> 
> Matteo Lanati
> Distributed Resources Group
> Leibniz-Rechenzentrum (LRZ)
> Boltzmannstrasse 1
> 85748	Garching b. München	(Germany)
> Phone: +49 89 35831 8724
> 

Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724

Re: HBase client with security

Posted by "Lanati, Matteo" <Ma...@lrz.de>.
Hi Devaraj,

you're right, I just subscribed, sorry for the spam.

Matteo


On Aug 29, 2013, at 3:31 PM, Devaraj k <de...@huawei.com> wrote:

> Please ask this question in user@hbase.apache.org, you would get better response there. 
> 
> Thanks
> Devaraj k
> 
> 
> -----Original Message-----
> From: Lanati, Matteo [mailto:Matteo.Lanati@lrz.de] 
> Sent: 29 August 2013 14:03
> To: <us...@hadoop.apache.org>
> Subject: HBase client with security
> 
> Hi all,
> 
> I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
> HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
> I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
> Regarding HBase client configuration:
> 
> - hbase-site.xml specifies
> 
>  <property>
>    <name>hbase.security.authentication</name>
>    <value>kerberos</value>
>  </property>
>  <property>
>    <name>hbase.rpc.engine</name>
>    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
>  </property>
>  <property>
>    <name>hbase.zookeeper.quorum</name>
>    <value>master.hadoop.local,host49.hadoop.local</value>
>  </property>
> 
> where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"
> 
> - I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.
> 
> Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log
> 
> 2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
> org.apache.hadoop.security.AccessControlException: Authentication is required
> 	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> 
> It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
> What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
> Thanks,
> 
> Matteo
> 
> 
> Matteo Lanati
> Distributed Resources Group
> Leibniz-Rechenzentrum (LRZ)
> Boltzmannstrasse 1
> 85748	Garching b. München	(Germany)
> Phone: +49 89 35831 8724
> 

Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724

Re: HBase client with security

Posted by "Lanati, Matteo" <Ma...@lrz.de>.
Hi Devaraj,

you're right, I just subscribed, sorry for the spam.

Matteo


On Aug 29, 2013, at 3:31 PM, Devaraj k <de...@huawei.com> wrote:

> Please ask this question in user@hbase.apache.org, you would get better response there. 
> 
> Thanks
> Devaraj k
> 
> 
> -----Original Message-----
> From: Lanati, Matteo [mailto:Matteo.Lanati@lrz.de] 
> Sent: 29 August 2013 14:03
> To: <us...@hadoop.apache.org>
> Subject: HBase client with security
> 
> Hi all,
> 
> I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
> HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
> I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
> Regarding HBase client configuration:
> 
> - hbase-site.xml specifies
> 
>  <property>
>    <name>hbase.security.authentication</name>
>    <value>kerberos</value>
>  </property>
>  <property>
>    <name>hbase.rpc.engine</name>
>    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
>  </property>
>  <property>
>    <name>hbase.zookeeper.quorum</name>
>    <value>master.hadoop.local,host49.hadoop.local</value>
>  </property>
> 
> where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"
> 
> - I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.
> 
> Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log
> 
> 2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
> org.apache.hadoop.security.AccessControlException: Authentication is required
> 	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> 
> It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
> What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
> Thanks,
> 
> Matteo
> 
> 
> Matteo Lanati
> Distributed Resources Group
> Leibniz-Rechenzentrum (LRZ)
> Boltzmannstrasse 1
> 85748	Garching b. München	(Germany)
> Phone: +49 89 35831 8724
> 

Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724

Re: HBase client with security

Posted by "Lanati, Matteo" <Ma...@lrz.de>.
Hi Devaraj,

you're right, I just subscribed, sorry for the spam.

Matteo


On Aug 29, 2013, at 3:31 PM, Devaraj k <de...@huawei.com> wrote:

> Please ask this question in user@hbase.apache.org, you would get better response there. 
> 
> Thanks
> Devaraj k
> 
> 
> -----Original Message-----
> From: Lanati, Matteo [mailto:Matteo.Lanati@lrz.de] 
> Sent: 29 August 2013 14:03
> To: <us...@hadoop.apache.org>
> Subject: HBase client with security
> 
> Hi all,
> 
> I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
> HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
> I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
> Regarding HBase client configuration:
> 
> - hbase-site.xml specifies
> 
>  <property>
>    <name>hbase.security.authentication</name>
>    <value>kerberos</value>
>  </property>
>  <property>
>    <name>hbase.rpc.engine</name>
>    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
>  </property>
>  <property>
>    <name>hbase.zookeeper.quorum</name>
>    <value>master.hadoop.local,host49.hadoop.local</value>
>  </property>
> 
> where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"
> 
> - I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.
> 
> Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log
> 
> 2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
> org.apache.hadoop.security.AccessControlException: Authentication is required
> 	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)
> 
> It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
> What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
> Thanks,
> 
> Matteo
> 
> 
> Matteo Lanati
> Distributed Resources Group
> Leibniz-Rechenzentrum (LRZ)
> Boltzmannstrasse 1
> 85748	Garching b. München	(Germany)
> Phone: +49 89 35831 8724
> 

Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724

RE: HBase client with security

Posted by Devaraj k <de...@huawei.com>.
Please ask this question in user@hbase.apache.org, you would get better response there. 

Thanks
Devaraj k


-----Original Message-----
From: Lanati, Matteo [mailto:Matteo.Lanati@lrz.de] 
Sent: 29 August 2013 14:03
To: <us...@hadoop.apache.org>
Subject: HBase client with security

Hi all,

I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
Regarding HBase client configuration:

- hbase-site.xml specifies

  <property>
    <name>hbase.security.authentication</name>
    <value>kerberos</value>
  </property>
  <property>
    <name>hbase.rpc.engine</name>
    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>master.hadoop.local,host49.hadoop.local</value>
  </property>

where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"

- I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.

Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log

2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
org.apache.hadoop.security.AccessControlException: Authentication is required
	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
Thanks,

Matteo


Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724

RE: HBase client with security

Posted by Devaraj k <de...@huawei.com>.
Please ask this question in user@hbase.apache.org, you would get better response there. 

Thanks
Devaraj k


-----Original Message-----
From: Lanati, Matteo [mailto:Matteo.Lanati@lrz.de] 
Sent: 29 August 2013 14:03
To: <us...@hadoop.apache.org>
Subject: HBase client with security

Hi all,

I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
Regarding HBase client configuration:

- hbase-site.xml specifies

  <property>
    <name>hbase.security.authentication</name>
    <value>kerberos</value>
  </property>
  <property>
    <name>hbase.rpc.engine</name>
    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>master.hadoop.local,host49.hadoop.local</value>
  </property>

where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"

- I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.

Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log

2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
org.apache.hadoop.security.AccessControlException: Authentication is required
	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
Thanks,

Matteo


Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724

RE: HBase client with security

Posted by Devaraj k <de...@huawei.com>.
Please ask this question in user@hbase.apache.org, you would get better response there. 

Thanks
Devaraj k


-----Original Message-----
From: Lanati, Matteo [mailto:Matteo.Lanati@lrz.de] 
Sent: 29 August 2013 14:03
To: <us...@hadoop.apache.org>
Subject: HBase client with security

Hi all,

I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
Regarding HBase client configuration:

- hbase-site.xml specifies

  <property>
    <name>hbase.security.authentication</name>
    <value>kerberos</value>
  </property>
  <property>
    <name>hbase.rpc.engine</name>
    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>master.hadoop.local,host49.hadoop.local</value>
  </property>

where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"

- I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.

Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log

2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
org.apache.hadoop.security.AccessControlException: Authentication is required
	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
Thanks,

Matteo


Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724

RE: HBase client with security

Posted by Devaraj k <de...@huawei.com>.
Please ask this question in user@hbase.apache.org, you would get better response there. 

Thanks
Devaraj k


-----Original Message-----
From: Lanati, Matteo [mailto:Matteo.Lanati@lrz.de] 
Sent: 29 August 2013 14:03
To: <us...@hadoop.apache.org>
Subject: HBase client with security

Hi all,

I set up Hadoop (1.2.0), Zookeeper (3.4.5) and HBase (0.94.8-security) with security.
HBase works if I launch the shell from the node running the master, but I'd like to use it from an external machine.
I prepared one, copying the Hadoop and HBase installation folders and adapting the path (indeed I can use the same client to run MR jobs and interact with HDFS).
Regarding HBase client configuration:

- hbase-site.xml specifies

  <property>
    <name>hbase.security.authentication</name>
    <value>kerberos</value>
  </property>
  <property>
    <name>hbase.rpc.engine</name>
    <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>master.hadoop.local,host49.hadoop.local</value>
  </property>

where the zookeeper hosts are reachable and can be solved via DNS. I had to specify them otherwise the shell complains about "org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid"

- I have a keytab for the principal I want to use (<user running hbase/my client hostname@MYREALM>), correctly addressed by the file hbase/conf/zk-jaas.conf. In hbase-env.sh, the variable HBASE_OPTS points to zk-jaas.conf.

Nonetheless, when I issue a command from a HBase shell on the client machine, I got an error in the HBase master log

2013-08-29 10:11:30,890 WARN org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: readAndProcess threw exception org.apache.hadoop.security.AccessControlException: Authentication is required. Count of bytes read: 0
org.apache.hadoop.security.AccessControlException: Authentication is required
	at org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:435)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:748)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:539)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:514)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

It looks like there's a mismatch between the client and the master regarding the authentication mechanism. Note that from the same client machine I can launch and use a Zookeeper shell.
What am I missing in the client configuration? Does /etc/krb5.conf play any role into this?
Thanks,

Matteo


Matteo Lanati
Distributed Resources Group
Leibniz-Rechenzentrum (LRZ)
Boltzmannstrasse 1
85748	Garching b. München	(Germany)
Phone: +49 89 35831 8724