You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Benjamin Kim <bb...@gmail.com> on 2016/09/02 21:46:59 UTC
Spark SQL Tables on top of HBase Tables
I was wondering if anyone has tried to create Spark SQL tables on top of HBase tables so that data in HBase can be accessed using Spark Thriftserver with SQL statements? This is similar what can be done using Hive.
Thanks,
Ben
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Spark SQL Tables on top of HBase Tables
Posted by Yan Zhou <yz...@gmail.com>.
There is a HSpark project, https://github.com/yzhou2001/HSpark, providing
native and fast access to HBase.
Currently it only supports Spark 1.4, but any suggestions and contributions
are more than welcome.
Try it out to find its speedups!
On Sat, Sep 3, 2016 at 12:57 PM, Mich Talebzadeh <mi...@gmail.com>
wrote:
> Mine is Hbase-0.98,
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 3 September 2016 at 20:51, Benjamin Kim <bb...@gmail.com> wrote:
>
>> I’m using Spark 1.6 and HBase 1.2. Have you got it to work using these
>> versions?
>>
>> On Sep 3, 2016, at 12:49 PM, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>> I am trying to find a solution for this
>>
>> ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class
>> org.apache.hadoop.hive.hbase.HBaseSerDe not found
>>
>> I am using Spark 2 and Hive 2!
>>
>> HTH
>>
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>> On 3 September 2016 at 20:31, Benjamin Kim <bb...@gmail.com> wrote:
>>
>>> Mich,
>>>
>>> I’m in the same boat. We can use Hive but not Spark.
>>>
>>> Cheers,
>>> Ben
>>>
>>> On Sep 2, 2016, at 3:37 PM, Mich Talebzadeh <mi...@gmail.com>
>>> wrote:
>>>
>>> Hi,
>>>
>>> You can create Hive external tables on top of existing Hbase table
>>> using the property
>>>
>>> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>>>
>>> Example
>>>
>>> hive> show create table hbase_table;
>>> OK
>>> CREATE TABLE `hbase_table`(
>>> `key` int COMMENT '',
>>> `value1` string COMMENT '',
>>> `value2` int COMMENT '',
>>> `value3` int COMMENT '')
>>> ROW FORMAT SERDE
>>> 'org.apache.hadoop.hive.hbase.HBaseSerDe'
>>> STORED BY
>>> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>>> WITH SERDEPROPERTIES (
>>> 'hbase.columns.mapping'=':key,a:b,a:c,d:e',
>>> 'serialization.format'='1')
>>> TBLPROPERTIES (
>>> 'transient_lastDdlTime'='1472370939')
>>>
>>> Then try to access this Hive table from Spark which is giving me grief
>>> at the moment :(
>>>
>>> scala> HiveContext.sql("use test")
>>> res9: org.apache.spark.sql.DataFrame = []
>>> scala> val hbase_table= spark.table("hbase_table")
>>> 16/09/02 23:31:07 ERROR log: error in initSerDe:
>>> java.lang.ClassNotFoundException Class org.apache.hadoop.hive.hbase.HBaseSerDe
>>> not found
>>>
>>> HTH
>>>
>>>
>>> Dr Mich Talebzadeh
>>>
>>>
>>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>>
>>>
>>> http://talebzadehmich.wordpress.com
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>> On 2 September 2016 at 23:08, KhajaAsmath Mohammed <
>>> mdkhajaasmath@gmail.com> wrote:
>>>
>>>> Hi Kim,
>>>>
>>>> I am also looking for same information. Just got the same requirement
>>>> today.
>>>>
>>>> Thanks,
>>>> Asmath
>>>>
>>>> On Fri, Sep 2, 2016 at 4:46 PM, Benjamin Kim <bb...@gmail.com>
>>>> wrote:
>>>>
>>>>> I was wondering if anyone has tried to create Spark SQL tables on top
>>>>> of HBase tables so that data in HBase can be accessed using Spark
>>>>> Thriftserver with SQL statements? This is similar what can be done using
>>>>> Hive.
>>>>>
>>>>> Thanks,
>>>>> Ben
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>
Re: Spark SQL Tables on top of HBase Tables
Posted by Mich Talebzadeh <mi...@gmail.com>.
Mine is Hbase-0.98,
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On 3 September 2016 at 20:51, Benjamin Kim <bb...@gmail.com> wrote:
> I’m using Spark 1.6 and HBase 1.2. Have you got it to work using these
> versions?
>
> On Sep 3, 2016, at 12:49 PM, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> I am trying to find a solution for this
>
> ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class
> org.apache.hadoop.hive.hbase.HBaseSerDe not found
>
> I am using Spark 2 and Hive 2!
>
> HTH
>
>
>
> Dr Mich Talebzadeh
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
> http://talebzadehmich.wordpress.com
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 3 September 2016 at 20:31, Benjamin Kim <bb...@gmail.com> wrote:
>
>> Mich,
>>
>> I’m in the same boat. We can use Hive but not Spark.
>>
>> Cheers,
>> Ben
>>
>> On Sep 2, 2016, at 3:37 PM, Mich Talebzadeh <mi...@gmail.com>
>> wrote:
>>
>> Hi,
>>
>> You can create Hive external tables on top of existing Hbase table using
>> the property
>>
>> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>>
>> Example
>>
>> hive> show create table hbase_table;
>> OK
>> CREATE TABLE `hbase_table`(
>> `key` int COMMENT '',
>> `value1` string COMMENT '',
>> `value2` int COMMENT '',
>> `value3` int COMMENT '')
>> ROW FORMAT SERDE
>> 'org.apache.hadoop.hive.hbase.HBaseSerDe'
>> STORED BY
>> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>> WITH SERDEPROPERTIES (
>> 'hbase.columns.mapping'=':key,a:b,a:c,d:e',
>> 'serialization.format'='1')
>> TBLPROPERTIES (
>> 'transient_lastDdlTime'='1472370939')
>>
>> Then try to access this Hive table from Spark which is giving me grief
>> at the moment :(
>>
>> scala> HiveContext.sql("use test")
>> res9: org.apache.spark.sql.DataFrame = []
>> scala> val hbase_table= spark.table("hbase_table")
>> 16/09/02 23:31:07 ERROR log: error in initSerDe:
>> java.lang.ClassNotFoundException Class org.apache.hadoop.hive.hbase.HBaseSerDe
>> not found
>>
>> HTH
>>
>>
>> Dr Mich Talebzadeh
>>
>>
>> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>>
>>
>> http://talebzadehmich.wordpress.com
>>
>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>> any loss, damage or destruction of data or any other property which may
>> arise from relying on this email's technical content is explicitly
>> disclaimed. The author will in no case be liable for any monetary damages
>> arising from such loss, damage or destruction.
>>
>>
>>
>> On 2 September 2016 at 23:08, KhajaAsmath Mohammed <
>> mdkhajaasmath@gmail.com> wrote:
>>
>>> Hi Kim,
>>>
>>> I am also looking for same information. Just got the same requirement
>>> today.
>>>
>>> Thanks,
>>> Asmath
>>>
>>> On Fri, Sep 2, 2016 at 4:46 PM, Benjamin Kim <bb...@gmail.com> wrote:
>>>
>>>> I was wondering if anyone has tried to create Spark SQL tables on top
>>>> of HBase tables so that data in HBase can be accessed using Spark
>>>> Thriftserver with SQL statements? This is similar what can be done using
>>>> Hive.
>>>>
>>>> Thanks,
>>>> Ben
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>>>
>>>>
>>>
>>
>>
>
>
Re: Spark SQL Tables on top of HBase Tables
Posted by Benjamin Kim <bb...@gmail.com>.
I’m using Spark 1.6 and HBase 1.2. Have you got it to work using these versions?
> On Sep 3, 2016, at 12:49 PM, Mich Talebzadeh <mi...@gmail.com> wrote:
>
> I am trying to find a solution for this
>
> ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
>
> I am using Spark 2 and Hive 2!
>
> HTH
>
>
>
>
> Dr Mich Talebzadeh
>
> LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>
>
> http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
>
> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
>
>
> On 3 September 2016 at 20:31, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
> Mich,
>
> I’m in the same boat. We can use Hive but not Spark.
>
> Cheers,
> Ben
>
>> On Sep 2, 2016, at 3:37 PM, Mich Talebzadeh <mich.talebzadeh@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hi,
>>
>> You can create Hive external tables on top of existing Hbase table using the property
>>
>> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>>
>> Example
>>
>> hive> show create table hbase_table;
>> OK
>> CREATE TABLE `hbase_table`(
>> `key` int COMMENT '',
>> `value1` string COMMENT '',
>> `value2` int COMMENT '',
>> `value3` int COMMENT '')
>> ROW FORMAT SERDE
>> 'org.apache.hadoop.hive.hbase.HBaseSerDe'
>> STORED BY
>> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>> WITH SERDEPROPERTIES (
>> 'hbase.columns.mapping'=':key,a:b,a:c,d:e',
>> 'serialization.format'='1')
>> TBLPROPERTIES (
>> 'transient_lastDdlTime'='1472370939')
>>
>> Then try to access this Hive table from Spark which is giving me grief at the moment :(
>>
>> scala> HiveContext.sql("use test")
>> res9: org.apache.spark.sql.DataFrame = []
>> scala> val hbase_table= spark.table("hbase_table")
>> 16/09/02 23:31:07 ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
>>
>> HTH
>>
>>
>> Dr Mich Talebzadeh
>>
>> LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>
>>
>> http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
>>
>> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
>>
>>
>> On 2 September 2016 at 23:08, KhajaAsmath Mohammed <mdkhajaasmath@gmail.com <ma...@gmail.com>> wrote:
>> Hi Kim,
>>
>> I am also looking for same information. Just got the same requirement today.
>>
>> Thanks,
>> Asmath
>>
>> On Fri, Sep 2, 2016 at 4:46 PM, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
>> I was wondering if anyone has tried to create Spark SQL tables on top of HBase tables so that data in HBase can be accessed using Spark Thriftserver with SQL statements? This is similar what can be done using Hive.
>>
>> Thanks,
>> Ben
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org <ma...@spark.apache.org>
>>
>>
>>
>
>
Re: Spark SQL Tables on top of HBase Tables
Posted by Mich Talebzadeh <mi...@gmail.com>.
I am trying to find a solution for this
ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class
org.apache.hadoop.hive.hbase.HBaseSerDe not found
I am using Spark 2 and Hive 2!
HTH
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On 3 September 2016 at 20:31, Benjamin Kim <bb...@gmail.com> wrote:
> Mich,
>
> I’m in the same boat. We can use Hive but not Spark.
>
> Cheers,
> Ben
>
> On Sep 2, 2016, at 3:37 PM, Mich Talebzadeh <mi...@gmail.com>
> wrote:
>
> Hi,
>
> You can create Hive external tables on top of existing Hbase table using
> the property
>
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>
> Example
>
> hive> show create table hbase_table;
> OK
> CREATE TABLE `hbase_table`(
> `key` int COMMENT '',
> `value1` string COMMENT '',
> `value2` int COMMENT '',
> `value3` int COMMENT '')
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.hbase.HBaseSerDe'
> STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> 'hbase.columns.mapping'=':key,a:b,a:c,d:e',
> 'serialization.format'='1')
> TBLPROPERTIES (
> 'transient_lastDdlTime'='1472370939')
>
> Then try to access this Hive table from Spark which is giving me grief at
> the moment :(
>
> scala> HiveContext.sql("use test")
> res9: org.apache.spark.sql.DataFrame = []
> scala> val hbase_table= spark.table("hbase_table")
> 16/09/02 23:31:07 ERROR log: error in initSerDe: java.lang.ClassNotFoundException
> Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
>
> HTH
>
>
> Dr Mich Talebzadeh
>
>
> LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
>
>
> http://talebzadehmich.wordpress.com
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
> On 2 September 2016 at 23:08, KhajaAsmath Mohammed <
> mdkhajaasmath@gmail.com> wrote:
>
>> Hi Kim,
>>
>> I am also looking for same information. Just got the same requirement
>> today.
>>
>> Thanks,
>> Asmath
>>
>> On Fri, Sep 2, 2016 at 4:46 PM, Benjamin Kim <bb...@gmail.com> wrote:
>>
>>> I was wondering if anyone has tried to create Spark SQL tables on top of
>>> HBase tables so that data in HBase can be accessed using Spark Thriftserver
>>> with SQL statements? This is similar what can be done using Hive.
>>>
>>> Thanks,
>>> Ben
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>>
>>>
>>
>
>
Re: Spark SQL Tables on top of HBase Tables
Posted by Benjamin Kim <bb...@gmail.com>.
Mich,
I’m in the same boat. We can use Hive but not Spark.
Cheers,
Ben
> On Sep 2, 2016, at 3:37 PM, Mich Talebzadeh <mi...@gmail.com> wrote:
>
> Hi,
>
> You can create Hive external tables on top of existing Hbase table using the property
>
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
>
> Example
>
> hive> show create table hbase_table;
> OK
> CREATE TABLE `hbase_table`(
> `key` int COMMENT '',
> `value1` string COMMENT '',
> `value2` int COMMENT '',
> `value3` int COMMENT '')
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.hbase.HBaseSerDe'
> STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES (
> 'hbase.columns.mapping'=':key,a:b,a:c,d:e',
> 'serialization.format'='1')
> TBLPROPERTIES (
> 'transient_lastDdlTime'='1472370939')
>
> Then try to access this Hive table from Spark which is giving me grief at the moment :(
>
> scala> HiveContext.sql("use test")
> res9: org.apache.spark.sql.DataFrame = []
> scala> val hbase_table= spark.table("hbase_table")
> 16/09/02 23:31:07 ERROR log: error in initSerDe: java.lang.ClassNotFoundException Class org.apache.hadoop.hive.hbase.HBaseSerDe not found
>
> HTH
>
>
> Dr Mich Talebzadeh
>
> LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>
>
> http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/>
>
> Disclaimer: Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
>
>
> On 2 September 2016 at 23:08, KhajaAsmath Mohammed <mdkhajaasmath@gmail.com <ma...@gmail.com>> wrote:
> Hi Kim,
>
> I am also looking for same information. Just got the same requirement today.
>
> Thanks,
> Asmath
>
> On Fri, Sep 2, 2016 at 4:46 PM, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
> I was wondering if anyone has tried to create Spark SQL tables on top of HBase tables so that data in HBase can be accessed using Spark Thriftserver with SQL statements? This is similar what can be done using Hive.
>
> Thanks,
> Ben
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org <ma...@spark.apache.org>
>
>
>
Re: Spark SQL Tables on top of HBase Tables
Posted by Mich Talebzadeh <mi...@gmail.com>.
Hi,
You can create Hive external tables on top of existing Hbase table using
the property
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
Example
hive> show create table hbase_table;
OK
CREATE TABLE `hbase_table`(
`key` int COMMENT '',
`value1` string COMMENT '',
`value2` int COMMENT '',
`value3` int COMMENT '')
ROW FORMAT SERDE
'org.apache.hadoop.hive.hbase.HBaseSerDe'
STORED BY
'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
'hbase.columns.mapping'=':key,a:b,a:c,d:e',
'serialization.format'='1')
TBLPROPERTIES (
'transient_lastDdlTime'='1472370939')
Then try to access this Hive table from Spark which is giving me grief at
the moment :(
scala> HiveContext.sql("use test")
res9: org.apache.spark.sql.DataFrame = []
scala> val hbase_table= spark.table("hbase_table")
16/09/02 23:31:07 ERROR log: error in initSerDe:
java.lang.ClassNotFoundException Class
org.apache.hadoop.hive.hbase.HBaseSerDe not found
HTH
Dr Mich Talebzadeh
LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On 2 September 2016 at 23:08, KhajaAsmath Mohammed <md...@gmail.com>
wrote:
> Hi Kim,
>
> I am also looking for same information. Just got the same requirement
> today.
>
> Thanks,
> Asmath
>
> On Fri, Sep 2, 2016 at 4:46 PM, Benjamin Kim <bb...@gmail.com> wrote:
>
>> I was wondering if anyone has tried to create Spark SQL tables on top of
>> HBase tables so that data in HBase can be accessed using Spark Thriftserver
>> with SQL statements? This is similar what can be done using Hive.
>>
>> Thanks,
>> Ben
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>
>>
>
Re: Spark SQL Tables on top of HBase Tables
Posted by ayan guha <gu...@gmail.com>.
You can either read hbase in rdd and then turn it to a df or expose hbase
tables using hive and read from hive or use phoenix
On 3 Sep 2016 08:08, "KhajaAsmath Mohammed" <md...@gmail.com> wrote:
> Hi Kim,
>
> I am also looking for same information. Just got the same requirement
> today.
>
> Thanks,
> Asmath
>
> On Fri, Sep 2, 2016 at 4:46 PM, Benjamin Kim <bb...@gmail.com> wrote:
>
>> I was wondering if anyone has tried to create Spark SQL tables on top of
>> HBase tables so that data in HBase can be accessed using Spark Thriftserver
>> with SQL statements? This is similar what can be done using Hive.
>>
>> Thanks,
>> Ben
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>
>>
>
Re: Spark SQL Tables on top of HBase Tables
Posted by KhajaAsmath Mohammed <md...@gmail.com>.
Hi Kim,
I am also looking for same information. Just got the same requirement today.
Thanks,
Asmath
On Fri, Sep 2, 2016 at 4:46 PM, Benjamin Kim <bb...@gmail.com> wrote:
> I was wondering if anyone has tried to create Spark SQL tables on top of
> HBase tables so that data in HBase can be accessed using Spark Thriftserver
> with SQL statements? This is similar what can be done using Hive.
>
> Thanks,
> Ben
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>