You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2019/01/28 18:09:00 UTC
[jira] [Resolved] (SPARK-26432) Not able to connect Hbase 2.1
service Getting NoSuchMethodException while trying to obtain token from
Hbase 2.1 service.
[ https://issues.apache.org/jira/browse/SPARK-26432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin resolved SPARK-26432.
------------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 23429
[https://github.com/apache/spark/pull/23429]
> Not able to connect Hbase 2.1 service Getting NoSuchMethodException while trying to obtain token from Hbase 2.1 service.
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-26432
> URL: https://issues.apache.org/jira/browse/SPARK-26432
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.3.2, 2.4.0
> Reporter: Sujith
> Assignee: Sujith
> Priority: Major
> Fix For: 3.0.0
>
> Attachments: hbase-dep-obtaintok.png
>
>
> Getting NoSuchMethodException :
> org.apache.hadoop.hbase.security.token.TokenUtil(org.apache.hadoop.conf.Configuration)
> while trying connect hbase 2.1 service from spark.
> This is mainly happening because in spark uses a deprecated hbase api
> public static Token<AuthenticationTokenIdentifier> obtainToken(Configuration conf)
> for obtaining the token and the same has been removed from hbase 2.1 version.
> Test steps:
> Steps to test Spark-Hbase connection
> 1. Create 2 tables in hbase shell
> >Launch hbase shell
> >Enter commands to create tables and load data
> create 'table1','cf'
> put 'table1','row1','cf:cid','20'
> create 'table2','cf'
> put 'table2','row1','cf:cid','30'
>
> >Show values command
> get 'table1','row1','cf:cid' will diplay value as 20
> get 'table2','row1','cf:cid' will diplay value as 30
>
>
> 2.Run SparkHbasetoHbase class in testSpark.jar using spark-submit
> spark-submit --master yarn-cluster --class com.mrs.example.spark.SparkHbasetoHbase --conf "spark.yarn.security.credentials.hbase.enabled"="true" --conf "spark.security.credentials.hbase.enabled"="true" --keytab /opt/client/user.keytab --principal sen testSpark.jar
> The SparkHbasetoHbase class will update the value of table2 with sum of values of table1 & table2.
> table2 = table1+table2
>
> 3.Verify the result in hbase shell
> Expected Result: The value of table2 should be 50.
> get 'table1','row1','cf:cid' will diplay value as 50
> Actual Result : Not updating the value as an error will be thrown when spark tries to connect with hbase service.
> Attached the snapshot of error logs below for more details
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org