You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "meiyoula (JIRA)" <ji...@apache.org> on 2015/05/22 03:52:17 UTC
[jira] [Comment Edited] (SPARK-7789) sql on security hbase:Token
generation only allowed for Kerberos authenticated clients
[ https://issues.apache.org/jira/browse/SPARK-7789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555435#comment-14555435 ]
meiyoula edited comment on SPARK-7789 at 5/22/15 1:51 AM:
----------------------------------------------------------
I also used hive 0.13 and Kerberos. [~deanchen]Has you executed the select statement. Below is my test sql statement.
{quote}
create table s1
(
key1 string,
c11 int,
c12 string,
c13 string,
c14 string
)
stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
with serdeproperties(
"hbase.columns.mapping" = ":key,
info:c11,
info:c12,
info:c13,
info:c14
")
tblproperties("hbase.table.name" = "shb1");
select * from s1;
{quote}
After reading the hive and hbase code, I think the root cause is that:
When the driver obtained the hbase token and add it into Credentials of CurrentUser, the hbase token will also go to executors. So the authentication of user(in executor) is TOKEN to hbase.But the hive code will send request to hbase sever to obtain token no matter what the authentication is. And the hbase code just allow the Kerberos authenticated clients to obtain token. So the exception occurs.
So I think the HIVE-8874 is meaningful, it should be merged.
was (Author: meiyoula):
I also used hive 0.13 and Kerberos. [~deanchen]Has you executed the select statement. Below is my test sql statement.
After reading the hive and hbase code, I think the root cause is that:
When the driver obtained the hbase token and add it into Credentials of CurrentUser, the hbase token will also go to executors. So the authentication of user(in executor) is TOKEN to hbase.But the hive code will send request to hbase sever to obtain token no matter what the authentication is. And the hbase code just allow the Kerberos authenticated clients to obtain token. So the exception occurs.
{quote}
create table s1
(
key1 string,
c11 int,
c12 string,
c13 string,
c14 string
)
stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
with serdeproperties(
"hbase.columns.mapping" = ":key,
info:c11,
info:c12,
info:c13,
info:c14
")
tblproperties("hbase.table.name" = "shb1");
select * from s1;
{quote}
So I think the HIVE-8874 is meaningful, it should be merged.
> sql on security hbase:Token generation only allowed for Kerberos authenticated clients
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-7789
> URL: https://issues.apache.org/jira/browse/SPARK-7789
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Reporter: meiyoula
>
> After creating a hbase table in beeline, then execute select sql statement, Executor occurs the exception:
> {quote}
> java.lang.IllegalStateException: Error while configuring input job properties
> at org.apache.hadoop.hive.hbase.HBaseStorageHandler.configureTableJobProperties(HBaseStorageHandler.java:343)
> at org.apache.hadoop.hive.hbase.HBaseStorageHandler.configureInputJobProperties(HBaseStorageHandler.java:279)
> at org.apache.hadoop.hive.ql.plan.PlanUtils.configureJobPropertiesForStorageHandler(PlanUtils.java:804)
> at org.apache.hadoop.hive.ql.plan.PlanUtils.configureInputJobPropertiesForStorageHandler(PlanUtils.java:774)
> at org.apache.spark.sql.hive.HadoopTableReader$.initializeLocalJobConfFunc(TableReader.scala:300)
> at org.apache.spark.sql.hive.HadoopTableReader$$anonfun$12.apply(TableReader.scala:276)
> at org.apache.spark.sql.hive.HadoopTableReader$$anonfun$12.apply(TableReader.scala:276)
> at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:176)
> at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:176)
> at scala.Option.map(Option.scala:145)
> at org.apache.spark.rdd.HadoopRDD.getJobConf(HadoopRDD.scala:176)
> at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:220)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:216)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
> at org.apache.spark.scheduler.Task.run(Task.scala:70)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.security.AccessDeniedException: org.apache.hadoop.hbase.security.AccessDeniedException: Token generation only allowed for Kerberos authenticated clients
> at org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:124)
> at org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$1.getAuthenticationToken(AuthenticationProtos.java:4267)
> at org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService.callMethod(AuthenticationProtos.java:4387)
> at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7696)
> at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1877)
> at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1859)
> at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32209)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2131)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
> at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
> at java.lang.Thread.run(Thread.java:745)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
> at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:326)
> at org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1636)
> at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92)
> at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89)
> at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
> at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95)
> at org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callBlockingMethod(CoprocessorRpcChannel.java:73)
> at org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$BlockingStub.getAuthenticationToken(AuthenticationProtos.java:4512)
> at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:86)
> at org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:111)
> at org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:108)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1672)
> at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:312)
> at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:108)
> at org.apache.hadoop.hbase.security.token.TokenUtil.obtainTokenForJob(TokenUtil.java:215)
> at org.apache.hadoop.hbase.security.token.TokenUtil.obtainTokenForJob(TokenUtil.java:196)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.hbase.util.Methods.call(Methods.java:39)
> at org.apache.hadoop.hbase.security.User$SecureHadoopUser.obtainAuthTokenForJob(User.java:321)
> at org.apache.hadoop.hive.hbase.HBaseStorageHandler.addHBaseDelegationToken(HBaseStorageHandler.java:371)
> at org.apache.hadoop.hive.hbase.HBaseStorageHandler.configureTableJobProperties(HBaseStorageHandler.java:340)
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org