You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by censj <ce...@lotuseed.com> on 2015/12/09 08:04:38 UTC

About Spark On Hbase

hi all,
     now I using spark,but I not found spark operation hbase open source. Do any one tell me? 
 

Re: About Spark On Hbase

Posted by Josh Mahonin <jm...@gmail.com>.
And as yet another option, there is
https://phoenix.apache.org/phoenix_spark.html

It however requires that you are also using Phoenix in conjunction with
HBase.

On Tue, Dec 15, 2015 at 4:16 PM, Ted Yu <yu...@gmail.com> wrote:

> There is also
> http://spark-packages.org/package/Huawei-Spark/Spark-SQL-on-HBase
>
> FYI
>
> On Tue, Dec 15, 2015 at 11:51 AM, Zhan Zhang <zz...@hortonworks.com>
> wrote:
>
>> If you want dataframe support, you can refer to
>> https://github.com/zhzhan/shc, which I am working on to integrate to
>> HBase upstream with existing support.
>>
>> Thanks.
>>
>> Zhan Zhang
>>
>> On Dec 15, 2015, at 4:34 AM, censj <ce...@lotuseed.com> wrote:
>>
>>
>> hi,*fight fate*
>> *Did I can in *bulkPut() function use Get value first ,then put this
>> value to Hbase ?
>>
>>
>> 在 2015年12月9日,16:02,censj <ce...@lotuseed.com> 写道:
>>
>> Thank you! I know!!!!
>>
>> 在 2015年12月9日,15:59,fightfate@163.com 写道:
>>
>> If you are using maven , you can add the cloudera maven repo to the
>> repository in pom.xml
>> and add the dependency of spark-hbase.
>> I just found this :
>> http://spark-packages.org/package/nerdammer/spark-hbase-connector
>> as Feng Dongyu recommend, you can try this also, but I had no experience
>> of using this.
>>
>>
>> ------------------------------
>> fightfate@163.com
>>
>>
>> *发件人:* censj <ce...@lotuseed.com>
>> *发送时间:* 2015-12-09 15:44
>> *收件人:* fightfate@163.com
>> *抄送:* user@spark.apache.org
>> *主题:* Re: About Spark On Hbase
>> So, I how to get this jar? I use set package project.I not found sbt lib.
>>
>> 在 2015年12月9日,15:42,fightfate@163.com 写道:
>>
>> I don't think it really need CDH component. Just use the API
>>
>> ------------------------------
>> fightfate@163.com
>>
>>
>> *发件人:* censj <ce...@lotuseed.com>
>> *发送时间:* 2015-12-09 15:31
>> *收件人:* fightfate@163.com
>> *抄送:* user@spark.apache.org
>> *主题:* Re: About Spark On Hbase
>> But this is dependent on CDH。I not install CDH。
>>
>> 在 2015年12月9日,15:18,fightfate@163.com 写道:
>>
>> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase
>> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992>
>> already integrates that feature into the hbase side, but
>> that feature has not been released.
>>
>> Best,
>> Sun.
>>
>> ------------------------------
>> fightfate@163.com
>>
>>
>> *From:* censj <ce...@lotuseed.com>
>> *Date:* 2015-12-09 15:04
>> *To:* user@spark.apache.org
>> *Subject:* About Spark On Hbase
>> hi all,
>>      now I using spark,but I not found spark operation hbase open
>> source. Do any one tell me?
>>
>>
>>
>>
>>
>

Re: About Spark On Hbase

Posted by Ted Yu <yu...@gmail.com>.
There is also
http://spark-packages.org/package/Huawei-Spark/Spark-SQL-on-HBase

FYI

On Tue, Dec 15, 2015 at 11:51 AM, Zhan Zhang <zz...@hortonworks.com> wrote:

> If you want dataframe support, you can refer to
> https://github.com/zhzhan/shc, which I am working on to integrate to
> HBase upstream with existing support.
>
> Thanks.
>
> Zhan Zhang
>
> On Dec 15, 2015, at 4:34 AM, censj <ce...@lotuseed.com> wrote:
>
>
> hi,*fight fate*
> *Did I can in *bulkPut() function use Get value first ,then put this
> value to Hbase ?
>
>
> 在 2015年12月9日,16:02,censj <ce...@lotuseed.com> 写道:
>
> Thank you! I know!!!!
>
> 在 2015年12月9日,15:59,fightfate@163.com 写道:
>
> If you are using maven , you can add the cloudera maven repo to the
> repository in pom.xml
> and add the dependency of spark-hbase.
> I just found this :
> http://spark-packages.org/package/nerdammer/spark-hbase-connector
> as Feng Dongyu recommend, you can try this also, but I had no experience
> of using this.
>
>
> ------------------------------
> fightfate@163.com
>
>
> *发件人:* censj <ce...@lotuseed.com>
> *发送时间:* 2015-12-09 15:44
> *收件人:* fightfate@163.com
> *抄送:* user@spark.apache.org
> *主题:* Re: About Spark On Hbase
> So, I how to get this jar? I use set package project.I not found sbt lib.
>
> 在 2015年12月9日,15:42,fightfate@163.com 写道:
>
> I don't think it really need CDH component. Just use the API
>
> ------------------------------
> fightfate@163.com
>
>
> *发件人:* censj <ce...@lotuseed.com>
> *发送时间:* 2015-12-09 15:31
> *收件人:* fightfate@163.com
> *抄送:* user@spark.apache.org
> *主题:* Re: About Spark On Hbase
> But this is dependent on CDH。I not install CDH。
>
> 在 2015年12月9日,15:18,fightfate@163.com 写道:
>
> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase
> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992>
> already integrates that feature into the hbase side, but
> that feature has not been released.
>
> Best,
> Sun.
>
> ------------------------------
> fightfate@163.com
>
>
> *From:* censj <ce...@lotuseed.com>
> *Date:* 2015-12-09 15:04
> *To:* user@spark.apache.org
> *Subject:* About Spark On Hbase
> hi all,
>      now I using spark,but I not found spark operation hbase open source.
> Do any one tell me?
>
>
>
>
>

Re: About Spark On Hbase

Posted by Zhan Zhang <zz...@hortonworks.com>.
If you want dataframe support, you can refer to https://github.com/zhzhan/shc, which I am working on to integrate to HBase upstream with existing support.

Thanks.

Zhan Zhang
On Dec 15, 2015, at 4:34 AM, censj <ce...@lotuseed.com>> wrote:


hi,fight fate
Did I can in bulkPut() function use Get value first ,then put this value to Hbase ?


在 2015年12月9日,16:02,censj <ce...@lotuseed.com>> 写道:

Thank you! I know!!!!
在 2015年12月9日,15:59,fightfate@163.com<ma...@163.com> 写道:

If you are using maven , you can add the cloudera maven repo to the repository in pom.xml
and add the dependency of spark-hbase.
I just found this : http://spark-packages.org/package/nerdammer/spark-hbase-connector
as Feng Dongyu recommend, you can try this also, but I had no experience of using this.


________________________________
fightfate@163.com<ma...@163.com>

发件人: censj<ma...@lotuseed.com>
发送时间: 2015-12-09 15:44
收件人: fightfate@163.com<ma...@163.com>
抄送: user@spark.apache.org<ma...@spark.apache.org>
主题: Re: About Spark On Hbase
So, I how to get this jar? I use set package project.I not found sbt lib.
在 2015年12月9日,15:42,fightfate@163.com<ma...@163.com> 写道:

I don't think it really need CDH component. Just use the API

________________________________
fightfate@163.com<ma...@163.com>

发件人: censj<ma...@lotuseed.com>
发送时间: 2015-12-09 15:31
收件人: fightfate@163.com<ma...@163.com>
抄送: user@spark.apache.org<ma...@spark.apache.org>
主题: Re: About Spark On Hbase
But this is dependent on CDH。I not install CDH。
在 2015年12月9日,15:18,fightfate@163.com<ma...@163.com> 写道:

Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase
Also, HBASE-13992<https://issues.apache.org/jira/browse/HBASE-13992>  already integrates that feature into the hbase side, but
that feature has not been released.

Best,
Sun.

________________________________
fightfate@163.com<ma...@163.com>

From: censj<ma...@lotuseed.com>
Date: 2015-12-09 15:04
To: user@spark.apache.org<ma...@spark.apache.org>
Subject: About Spark On Hbase
hi all,
     now I using spark,but I not found spark operation hbase open source. Do any one tell me?




Re: About Spark On Hbase

Posted by censj <ce...@lotuseed.com>.
hi,fight fate
	Did I can in bulkPut() function use Get value first ,then put this value to Hbase ?


> 在 2015年12月9日,16:02,censj <ce...@lotuseed.com> 写道:
> 
> Thank you! I know!!!!
>> 在 2015年12月9日,15:59,fightfate@163.com <ma...@163.com> 写道:
>> 
>> If you are using maven , you can add the cloudera maven repo to the repository in pom.xml 
>> and add the dependency of spark-hbase. 
>> I just found this : http://spark-packages.org/package/nerdammer/spark-hbase-connector <http://spark-packages.org/package/nerdammer/spark-hbase-connector> 
>> as Feng Dongyu recommend, you can try this also, but I had no experience of using this. 
>> 
>> 
>> fightfate@163.com <ma...@163.com>
>>  
>> 发件人: censj <ma...@lotuseed.com>
>> 发送时间: 2015-12-09 15:44
>> 收件人: fightfate@163.com <ma...@163.com>
>> 抄送: user@spark.apache.org <ma...@spark.apache.org>
>> 主题: Re: About Spark On Hbase
>> So, I how to get this jar? I use set package project.I not found sbt lib.
>>> 在 2015年12月9日,15:42,fightfate@163.com <ma...@163.com> 写道:
>>> 
>>> I don't think it really need CDH component. Just use the API 
>>> 
>>> fightfate@163.com <ma...@163.com>
>>>  
>>> 发件人: censj <ma...@lotuseed.com>
>>> 发送时间: 2015-12-09 15:31
>>> 收件人: fightfate@163.com <ma...@163.com>
>>> 抄送: user@spark.apache.org <ma...@spark.apache.org>
>>> 主题: Re: About Spark On Hbase
>>> But this is dependent on CDH。I not install CDH。
>>>> 在 2015年12月9日,15:18,fightfate@163.com <ma...@163.com> 写道:
>>>> 
>>>> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase <https://github.com/cloudera-labs/SparkOnHBase> 
>>>> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992>  already integrates that feature into the hbase side, but 
>>>> that feature has not been released. 
>>>> 
>>>> Best,
>>>> Sun.
>>>> 
>>>> fightfate@163.com <ma...@163.com>
>>>>  
>>>> From: censj <ma...@lotuseed.com>
>>>> Date: 2015-12-09 15:04
>>>> To: user@spark.apache.org <ma...@spark.apache.org>
>>>> Subject: About Spark On Hbase
>>>> hi all,
>>>>      now I using spark,but I not found spark operation hbase open source. Do any one tell me? 
> 


Re: About Spark On Hbase

Posted by censj <ce...@lotuseed.com>.
Thank you! I know!!!!
> 在 2015年12月9日,15:59,fightfate@163.com 写道:
> 
> If you are using maven , you can add the cloudera maven repo to the repository in pom.xml 
> and add the dependency of spark-hbase. 
> I just found this : http://spark-packages.org/package/nerdammer/spark-hbase-connector <http://spark-packages.org/package/nerdammer/spark-hbase-connector> 
> as Feng Dongyu recommend, you can try this also, but I had no experience of using this. 
> 
> 
> fightfate@163.com <ma...@163.com>
>  
> 发件人: censj <ma...@lotuseed.com>
> 发送时间: 2015-12-09 15:44
> 收件人: fightfate@163.com <ma...@163.com>
> 抄送: user@spark.apache.org <ma...@spark.apache.org>
> 主题: Re: About Spark On Hbase
> So, I how to get this jar? I use set package project.I not found sbt lib.
>> 在 2015年12月9日,15:42,fightfate@163.com <ma...@163.com> 写道:
>> 
>> I don't think it really need CDH component. Just use the API 
>> 
>> fightfate@163.com <ma...@163.com>
>>  
>> 发件人: censj <ma...@lotuseed.com>
>> 发送时间: 2015-12-09 15:31
>> 收件人: fightfate@163.com <ma...@163.com>
>> 抄送: user@spark.apache.org <ma...@spark.apache.org>
>> 主题: Re: About Spark On Hbase
>> But this is dependent on CDH。I not install CDH。
>>> 在 2015年12月9日,15:18,fightfate@163.com <ma...@163.com> 写道:
>>> 
>>> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase <https://github.com/cloudera-labs/SparkOnHBase> 
>>> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992>  already integrates that feature into the hbase side, but 
>>> that feature has not been released. 
>>> 
>>> Best,
>>> Sun.
>>> 
>>> fightfate@163.com <ma...@163.com>
>>>  
>>> From: censj <ma...@lotuseed.com>
>>> Date: 2015-12-09 15:04
>>> To: user@spark.apache.org <ma...@spark.apache.org>
>>> Subject: About Spark On Hbase
>>> hi all,
>>>      now I using spark,but I not found spark operation hbase open source. Do any one tell me? 


回复: Re: About Spark On Hbase

Posted by "fightfate@163.com" <fi...@163.com>.
If you are using maven , you can add the cloudera maven repo to the repository in pom.xml 
and add the dependency of spark-hbase. 
I just found this : http://spark-packages.org/package/nerdammer/spark-hbase-connector 
as Feng Dongyu recommend, you can try this also, but I had no experience of using this. 




fightfate@163.com
 
发件人: censj
发送时间: 2015-12-09 15:44
收件人: fightfate@163.com
抄送: user@spark.apache.org
主题: Re: About Spark On Hbase
So, I how to get this jar? I use set package project.I not found sbt lib.
在 2015年12月9日,15:42,fightfate@163.com 写道:

I don't think it really need CDH component. Just use the API 



fightfate@163.com
 
发件人: censj
发送时间: 2015-12-09 15:31
收件人: fightfate@163.com
抄送: user@spark.apache.org
主题: Re: About Spark On Hbase
But this is dependent on CDH。I not install CDH。
在 2015年12月9日,15:18,fightfate@163.com 写道:

Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase 
Also, HBASE-13992  already integrates that feature into the hbase side, but 
that feature has not been released. 

Best,
Sun.



fightfate@163.com
 
From: censj
Date: 2015-12-09 15:04
To: user@spark.apache.org
Subject: About Spark On Hbase
hi all,
     now I using spark,but I not found spark operation hbase open source. Do any one tell me? 


Re: About Spark On Hbase

Posted by censj <ce...@lotuseed.com>.
So, I how to get this jar? I use set package project.I not found sbt lib.
> 在 2015年12月9日,15:42,fightfate@163.com 写道:
> 
> I don't think it really need CDH component. Just use the API 
> 
> fightfate@163.com <ma...@163.com>
>  
> 发件人: censj <ma...@lotuseed.com>
> 发送时间: 2015-12-09 15:31
> 收件人: fightfate@163.com <ma...@163.com>
> 抄送: user@spark.apache.org <ma...@spark.apache.org>
> 主题: Re: About Spark On Hbase
> But this is dependent on CDH。I not install CDH。
>> 在 2015年12月9日,15:18,fightfate@163.com <ma...@163.com> 写道:
>> 
>> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase <https://github.com/cloudera-labs/SparkOnHBase> 
>> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992>  already integrates that feature into the hbase side, but 
>> that feature has not been released. 
>> 
>> Best,
>> Sun.
>> 
>> fightfate@163.com <ma...@163.com>
>>  
>> From: censj <ma...@lotuseed.com>
>> Date: 2015-12-09 15:04
>> To: user@spark.apache.org <ma...@spark.apache.org>
>> Subject: About Spark On Hbase
>> hi all,
>>      now I using spark,but I not found spark operation hbase open source. Do any one tell me? 


回复: Re: About Spark On Hbase

Posted by "fightfate@163.com" <fi...@163.com>.
I don't think it really need CDH component. Just use the API 



fightfate@163.com
 
发件人: censj
发送时间: 2015-12-09 15:31
收件人: fightfate@163.com
抄送: user@spark.apache.org
主题: Re: About Spark On Hbase
But this is dependent on CDH。I not install CDH。
在 2015年12月9日,15:18,fightfate@163.com 写道:

Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase 
Also, HBASE-13992  already integrates that feature into the hbase side, but 
that feature has not been released. 

Best,
Sun.



fightfate@163.com
 
From: censj
Date: 2015-12-09 15:04
To: user@spark.apache.org
Subject: About Spark On Hbase
hi all,
     now I using spark,but I not found spark operation hbase open source. Do any one tell me? 


Re: About Spark On Hbase

Posted by censj <ce...@lotuseed.com>.
But this is dependent on CDH。I not install CDH。
> 在 2015年12月9日,15:18,fightfate@163.com 写道:
> 
> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase <https://github.com/cloudera-labs/SparkOnHBase> 
> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992>  already integrates that feature into the hbase side, but 
> that feature has not been released. 
> 
> Best,
> Sun.
> 
> fightfate@163.com <ma...@163.com>
>  
> From: censj <ma...@lotuseed.com>
> Date: 2015-12-09 15:04
> To: user@spark.apache.org <ma...@spark.apache.org>
> Subject: About Spark On Hbase
> hi all,
>      now I using spark,but I not found spark operation hbase open source. Do any one tell me? 


Re: About Spark On Hbase

Posted by "fightfate@163.com" <fi...@163.com>.
Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase 
Also, HBASE-13992  already integrates that feature into the hbase side, but 
that feature has not been released. 

Best,
Sun.



fightfate@163.com
 
From: censj
Date: 2015-12-09 15:04
To: user@spark.apache.org
Subject: About Spark On Hbase
hi all,
     now I using spark,but I not found spark operation hbase open source. Do any one tell me? 
 

Re: About Spark On Hbase

Posted by censj <ce...@lotuseed.com>.
Can you get me a example?
I want to update base data.
> 在 2015年12月9日,15:19,Fengdong Yu <fe...@everstring.com> 写道:
> 
> https://github.com/nerdammer/spark-hbase-connector <https://github.com/nerdammer/spark-hbase-connector>
> 
> This is better and easy to use.
> 
> 
> 
> 
> 
>> On Dec 9, 2015, at 3:04 PM, censj <censj@lotuseed.com <ma...@lotuseed.com>> wrote:
>> 
>> hi all,
>>      now I using spark,but I not found spark operation hbase open source. Do any one tell me? 
>>  
> 


Re: About Spark On Hbase

Posted by Fengdong Yu <fe...@everstring.com>.
https://github.com/nerdammer/spark-hbase-connector

This is better and easy to use.





> On Dec 9, 2015, at 3:04 PM, censj <ce...@lotuseed.com> wrote:
> 
> hi all,
>      now I using spark,but I not found spark operation hbase open source. Do any one tell me? 
>