You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Suhas Satish <su...@gmail.com> on 2013/08/05 21:16:19 UTC
sqoop import into secure Hbase with kerberos
I have configured hbase 94.9 with kerberos successfully for authentication
and authorization as mentioned in the CDH security docs. I am using sqoop
1.4.3. Is there any configuration required from the sqoop client side for
kerberos?
I have the following permissions on hbase tables -
hbase(main):003:0> grant 'kuser1', 'RWXCA', 'demo'
ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
permissions (user=kuser1, scope=demo, family=, qualifer=, action=ADMIN)
bin/sqoop import --connect jdbc:mysql://10.10.1.10/TestDB --table t1
--hbase-table t1 --column-family world
When I try to import into it using sqoop with the above cmd, I get the
following error -
2013-08-05 11:59:33,121 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer:
org.apache.hadoop.hbase.security.AccessDeniedException: Token generation
only allowed for Kerberos authenticated clients
at
org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:5576)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3868)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
Thanks,
Suhas.
Re: sqoop import into secure Hbase with kerberos
Posted by ssatish <su...@gmail.com>.
Thanks for your suggestions. I was able to successfully scan tables from the
hbase side. So there's no problem in hbase kerberos configuration. Sqoop is
not picking up the correct kerberos credentials.
Scan _acl_ also worked -
demo column=l:kuser1, timestamp=1375737971153,
value=RWXCA
testtable column=l:kuser1, timestamp=1375810415055,
value=RWXCA
2 row(s) in 0.0380 seconds
I have to look at Sqoop to see if it has the following 2 method calls -
HBaseConfiguration.merge(conf, HBaseConfiguration.create(conf));
TableMapReduceUtil.initCredentials(job);
Thanks once again.
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/sqoop-import-into-secure-Hbase-with-kerberos-tp4048847p4048911.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: sqoop import into secure Hbase with kerberos
Posted by anil gupta <an...@gmail.com>.
In addition to Gary's suggestion.
Try to scan "_acl_" table and check that kuser1 should have proper
permissions.
I haven't used Sqoop for loading data directly into HBase. But, i have used
MR to load data into HBase. So, you might have to look at the way the MR
job from sqoop is set-up.
These two methods should be present in the MR job set-up for HBase:
HBaseConfiguration.merge(conf, HBaseConfiguration.create(conf));
TableMapReduceUtil.initCredentials(job);
If the above two method calls are missing then Sqoop might have problems in
loading data. Maybe you can ask this on Sqoop mailing list.
On Mon, Aug 5, 2013 at 6:33 PM, Gary Helmling <gh...@gmail.com> wrote:
> To further isolate the problem, try doing some simple commands from the
> hbase shell after obtaining kerberos credentials:
>
> 1) kinit
> 2) hbase shell
> 3) in hbase shell:
> create 'testtable', 'f'
> put 'testtable', 'r1', 'f:col1', 'val1'
> get 'testtable', 'r1'
>
> If these all work, then the HBase code is correctly recognizing and
> utilizing your kerberos credentials, and there may be a problem with
> getting the credentials passed through the sqoop process. If these
> commands do _not_ work, then there is a problem with your kerberos / HBase
> security setup. For example. it could be a mismatch in the encryption
> ciphers (most kerberos configs will default to AES which is not or at least
> was not previously supported in Java 6 without the "unlimited strength" JCE
> policy files).
>
>
> On Mon, Aug 5, 2013 at 6:14 PM, ssatish <su...@gmail.com> wrote:
>
> > I got the grant working for user by adding kuser1 to sudo users list by
> > adding the following code to hbase-site.xml and restarting master and
> > region
> > servers.
> >
> > hbase.superuser
> > kuser1
> >
> > But I still get the ImportTool error - ERROR tool.ImportTool: Error
> during
> > import: Can't get authentication token
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-hbase.679495.n3.nabble.com/sqoop-import-into-secure-Hbase-with-kerberos-tp4048847p4048864.html
> > Sent from the HBase User mailing list archive at Nabble.com.
> >
>
--
Thanks & Regards,
Anil Gupta
Re: sqoop import into secure Hbase with kerberos
Posted by Gary Helmling <gh...@gmail.com>.
To further isolate the problem, try doing some simple commands from the
hbase shell after obtaining kerberos credentials:
1) kinit
2) hbase shell
3) in hbase shell:
create 'testtable', 'f'
put 'testtable', 'r1', 'f:col1', 'val1'
get 'testtable', 'r1'
If these all work, then the HBase code is correctly recognizing and
utilizing your kerberos credentials, and there may be a problem with
getting the credentials passed through the sqoop process. If these
commands do _not_ work, then there is a problem with your kerberos / HBase
security setup. For example. it could be a mismatch in the encryption
ciphers (most kerberos configs will default to AES which is not or at least
was not previously supported in Java 6 without the "unlimited strength" JCE
policy files).
On Mon, Aug 5, 2013 at 6:14 PM, ssatish <su...@gmail.com> wrote:
> I got the grant working for user by adding kuser1 to sudo users list by
> adding the following code to hbase-site.xml and restarting master and
> region
> servers.
>
> hbase.superuser
> kuser1
>
> But I still get the ImportTool error - ERROR tool.ImportTool: Error during
> import: Can't get authentication token
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/sqoop-import-into-secure-Hbase-with-kerberos-tp4048847p4048864.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
Re: sqoop import into secure Hbase with kerberos
Posted by ssatish <su...@gmail.com>.
I got the grant working for user by adding kuser1 to sudo users list by
adding the following code to hbase-site.xml and restarting master and region
servers.
hbase.superuser
kuser1
But I still get the ImportTool error - ERROR tool.ImportTool: Error during
import: Can't get authentication token
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/sqoop-import-into-secure-Hbase-with-kerberos-tp4048847p4048864.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: sqoop import into secure Hbase with kerberos
Posted by Ted Yu <yu...@gmail.com>.
> hbase(main):003:0> grant 'kuser1', 'RWXCA', 'demo'
> ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
> permissions (user=kuser1, scope=demo, family=, qualifer=, action=ADMIN)
The grant failed.
Did you get it working ?
On Mon, Aug 5, 2013 at 6:02 PM, ssatish <su...@gmail.com> wrote:
> Yes klist output shows valid credentials
>
> Ticket cache: FILE:/tmp/krb5cc_0
> Default principal: kuser1@QA.LAB
>
> Valid starting Expires Service principal
> 08/05/13 12:34:42 08/05/13 22:34:42 krbtgt/QA.LAB@QA.LAB
> renew until 08/05/13 12:34:42
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/sqoop-import-into-secure-Hbase-with-kerberos-tp4048847p4048862.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
Re: sqoop import into secure Hbase with kerberos
Posted by ssatish <su...@gmail.com>.
Yes klist output shows valid credentials
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: kuser1@QA.LAB
Valid starting Expires Service principal
08/05/13 12:34:42 08/05/13 22:34:42 krbtgt/QA.LAB@QA.LAB
renew until 08/05/13 12:34:42
--
View this message in context: http://apache-hbase.679495.n3.nabble.com/sqoop-import-into-secure-Hbase-with-kerberos-tp4048847p4048862.html
Sent from the HBase User mailing list archive at Nabble.com.
Re: sqoop import into secure Hbase with kerberos
Posted by Anil Gupta <an...@gmail.com>.
Are you running kinit command before sqoop?
If yes, then check running klist and see whether the user is logged in or not.
Best Regards,
Anil
On Aug 5, 2013, at 12:16 PM, Suhas Satish <su...@gmail.com> wrote:
> I have configured hbase 94.9 with kerberos successfully for authentication
> and authorization as mentioned in the CDH security docs. I am using sqoop
> 1.4.3. Is there any configuration required from the sqoop client side for
> kerberos?
>
> I have the following permissions on hbase tables -
> hbase(main):003:0> grant 'kuser1', 'RWXCA', 'demo'
> ERROR: org.apache.hadoop.hbase.security.AccessDeniedException:
> org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient
> permissions (user=kuser1, scope=demo, family=, qualifer=, action=ADMIN)
>
>
> bin/sqoop import --connect jdbc:mysql://10.10.1.10/TestDB --table t1
> --hbase-table t1 --column-family world
>
>
> When I try to import into it using sqoop with the above cmd, I get the
> following error -
>
>
> 2013-08-05 11:59:33,121 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> org.apache.hadoop.hbase.security.AccessDeniedException: Token generation
> only allowed for Kerberos authenticated clients
> at
> org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java:5576)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java:3868)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java:308)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
>
>
> Thanks,
> Suhas.