You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by censj <ce...@lotuseed.com> on 2015/12/09 08:04:38 UTC
About Spark On Hbase
hi all,
now I using spark,but I not found spark operation hbase open source. Do any one tell me?
Re: About Spark On Hbase
Posted by Josh Mahonin <jm...@gmail.com>.
And as yet another option, there is
https://phoenix.apache.org/phoenix_spark.html
It however requires that you are also using Phoenix in conjunction with
HBase.
On Tue, Dec 15, 2015 at 4:16 PM, Ted Yu <yu...@gmail.com> wrote:
> There is also
> http://spark-packages.org/package/Huawei-Spark/Spark-SQL-on-HBase
>
> FYI
>
> On Tue, Dec 15, 2015 at 11:51 AM, Zhan Zhang <zz...@hortonworks.com>
> wrote:
>
>> If you want dataframe support, you can refer to
>> https://github.com/zhzhan/shc, which I am working on to integrate to
>> HBase upstream with existing support.
>>
>> Thanks.
>>
>> Zhan Zhang
>>
>> On Dec 15, 2015, at 4:34 AM, censj <ce...@lotuseed.com> wrote:
>>
>>
>> hi,*fight fate*
>> *Did I can in *bulkPut() function use Get value first ,then put this
>> value to Hbase ?
>>
>>
>> 在 2015年12月9日,16:02,censj <ce...@lotuseed.com> 写道:
>>
>> Thank you! I know!!!!
>>
>> 在 2015年12月9日,15:59,fightfate@163.com 写道:
>>
>> If you are using maven , you can add the cloudera maven repo to the
>> repository in pom.xml
>> and add the dependency of spark-hbase.
>> I just found this :
>> http://spark-packages.org/package/nerdammer/spark-hbase-connector
>> as Feng Dongyu recommend, you can try this also, but I had no experience
>> of using this.
>>
>>
>> ------------------------------
>> fightfate@163.com
>>
>>
>> *发件人:* censj <ce...@lotuseed.com>
>> *发送时间:* 2015-12-09 15:44
>> *收件人:* fightfate@163.com
>> *抄送:* user@spark.apache.org
>> *主题:* Re: About Spark On Hbase
>> So, I how to get this jar? I use set package project.I not found sbt lib.
>>
>> 在 2015年12月9日,15:42,fightfate@163.com 写道:
>>
>> I don't think it really need CDH component. Just use the API
>>
>> ------------------------------
>> fightfate@163.com
>>
>>
>> *发件人:* censj <ce...@lotuseed.com>
>> *发送时间:* 2015-12-09 15:31
>> *收件人:* fightfate@163.com
>> *抄送:* user@spark.apache.org
>> *主题:* Re: About Spark On Hbase
>> But this is dependent on CDH。I not install CDH。
>>
>> 在 2015年12月9日,15:18,fightfate@163.com 写道:
>>
>> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase
>> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992>
>> already integrates that feature into the hbase side, but
>> that feature has not been released.
>>
>> Best,
>> Sun.
>>
>> ------------------------------
>> fightfate@163.com
>>
>>
>> *From:* censj <ce...@lotuseed.com>
>> *Date:* 2015-12-09 15:04
>> *To:* user@spark.apache.org
>> *Subject:* About Spark On Hbase
>> hi all,
>> now I using spark,but I not found spark operation hbase open
>> source. Do any one tell me?
>>
>>
>>
>>
>>
>
Re: About Spark On Hbase
Posted by Ted Yu <yu...@gmail.com>.
There is also
http://spark-packages.org/package/Huawei-Spark/Spark-SQL-on-HBase
FYI
On Tue, Dec 15, 2015 at 11:51 AM, Zhan Zhang <zz...@hortonworks.com> wrote:
> If you want dataframe support, you can refer to
> https://github.com/zhzhan/shc, which I am working on to integrate to
> HBase upstream with existing support.
>
> Thanks.
>
> Zhan Zhang
>
> On Dec 15, 2015, at 4:34 AM, censj <ce...@lotuseed.com> wrote:
>
>
> hi,*fight fate*
> *Did I can in *bulkPut() function use Get value first ,then put this
> value to Hbase ?
>
>
> 在 2015年12月9日,16:02,censj <ce...@lotuseed.com> 写道:
>
> Thank you! I know!!!!
>
> 在 2015年12月9日,15:59,fightfate@163.com 写道:
>
> If you are using maven , you can add the cloudera maven repo to the
> repository in pom.xml
> and add the dependency of spark-hbase.
> I just found this :
> http://spark-packages.org/package/nerdammer/spark-hbase-connector
> as Feng Dongyu recommend, you can try this also, but I had no experience
> of using this.
>
>
> ------------------------------
> fightfate@163.com
>
>
> *发件人:* censj <ce...@lotuseed.com>
> *发送时间:* 2015-12-09 15:44
> *收件人:* fightfate@163.com
> *抄送:* user@spark.apache.org
> *主题:* Re: About Spark On Hbase
> So, I how to get this jar? I use set package project.I not found sbt lib.
>
> 在 2015年12月9日,15:42,fightfate@163.com 写道:
>
> I don't think it really need CDH component. Just use the API
>
> ------------------------------
> fightfate@163.com
>
>
> *发件人:* censj <ce...@lotuseed.com>
> *发送时间:* 2015-12-09 15:31
> *收件人:* fightfate@163.com
> *抄送:* user@spark.apache.org
> *主题:* Re: About Spark On Hbase
> But this is dependent on CDH。I not install CDH。
>
> 在 2015年12月9日,15:18,fightfate@163.com 写道:
>
> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase
> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992>
> already integrates that feature into the hbase side, but
> that feature has not been released.
>
> Best,
> Sun.
>
> ------------------------------
> fightfate@163.com
>
>
> *From:* censj <ce...@lotuseed.com>
> *Date:* 2015-12-09 15:04
> *To:* user@spark.apache.org
> *Subject:* About Spark On Hbase
> hi all,
> now I using spark,but I not found spark operation hbase open source.
> Do any one tell me?
>
>
>
>
>
Re: About Spark On Hbase
Posted by Zhan Zhang <zz...@hortonworks.com>.
If you want dataframe support, you can refer to https://github.com/zhzhan/shc, which I am working on to integrate to HBase upstream with existing support.
Thanks.
Zhan Zhang
On Dec 15, 2015, at 4:34 AM, censj <ce...@lotuseed.com>> wrote:
hi,fight fate
Did I can in bulkPut() function use Get value first ,then put this value to Hbase ?
在 2015年12月9日,16:02,censj <ce...@lotuseed.com>> 写道:
Thank you! I know!!!!
在 2015年12月9日,15:59,fightfate@163.com<ma...@163.com> 写道:
If you are using maven , you can add the cloudera maven repo to the repository in pom.xml
and add the dependency of spark-hbase.
I just found this : http://spark-packages.org/package/nerdammer/spark-hbase-connector
as Feng Dongyu recommend, you can try this also, but I had no experience of using this.
________________________________
fightfate@163.com<ma...@163.com>
发件人: censj<ma...@lotuseed.com>
发送时间: 2015-12-09 15:44
收件人: fightfate@163.com<ma...@163.com>
抄送: user@spark.apache.org<ma...@spark.apache.org>
主题: Re: About Spark On Hbase
So, I how to get this jar? I use set package project.I not found sbt lib.
在 2015年12月9日,15:42,fightfate@163.com<ma...@163.com> 写道:
I don't think it really need CDH component. Just use the API
________________________________
fightfate@163.com<ma...@163.com>
发件人: censj<ma...@lotuseed.com>
发送时间: 2015-12-09 15:31
收件人: fightfate@163.com<ma...@163.com>
抄送: user@spark.apache.org<ma...@spark.apache.org>
主题: Re: About Spark On Hbase
But this is dependent on CDH。I not install CDH。
在 2015年12月9日,15:18,fightfate@163.com<ma...@163.com> 写道:
Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase
Also, HBASE-13992<https://issues.apache.org/jira/browse/HBASE-13992> already integrates that feature into the hbase side, but
that feature has not been released.
Best,
Sun.
________________________________
fightfate@163.com<ma...@163.com>
From: censj<ma...@lotuseed.com>
Date: 2015-12-09 15:04
To: user@spark.apache.org<ma...@spark.apache.org>
Subject: About Spark On Hbase
hi all,
now I using spark,but I not found spark operation hbase open source. Do any one tell me?
Re: About Spark On Hbase
Posted by censj <ce...@lotuseed.com>.
hi,fight fate
Did I can in bulkPut() function use Get value first ,then put this value to Hbase ?
> 在 2015年12月9日,16:02,censj <ce...@lotuseed.com> 写道:
>
> Thank you! I know!!!!
>> 在 2015年12月9日,15:59,fightfate@163.com <ma...@163.com> 写道:
>>
>> If you are using maven , you can add the cloudera maven repo to the repository in pom.xml
>> and add the dependency of spark-hbase.
>> I just found this : http://spark-packages.org/package/nerdammer/spark-hbase-connector <http://spark-packages.org/package/nerdammer/spark-hbase-connector>
>> as Feng Dongyu recommend, you can try this also, but I had no experience of using this.
>>
>>
>> fightfate@163.com <ma...@163.com>
>>
>> 发件人: censj <ma...@lotuseed.com>
>> 发送时间: 2015-12-09 15:44
>> 收件人: fightfate@163.com <ma...@163.com>
>> 抄送: user@spark.apache.org <ma...@spark.apache.org>
>> 主题: Re: About Spark On Hbase
>> So, I how to get this jar? I use set package project.I not found sbt lib.
>>> 在 2015年12月9日,15:42,fightfate@163.com <ma...@163.com> 写道:
>>>
>>> I don't think it really need CDH component. Just use the API
>>>
>>> fightfate@163.com <ma...@163.com>
>>>
>>> 发件人: censj <ma...@lotuseed.com>
>>> 发送时间: 2015-12-09 15:31
>>> 收件人: fightfate@163.com <ma...@163.com>
>>> 抄送: user@spark.apache.org <ma...@spark.apache.org>
>>> 主题: Re: About Spark On Hbase
>>> But this is dependent on CDH。I not install CDH。
>>>> 在 2015年12月9日,15:18,fightfate@163.com <ma...@163.com> 写道:
>>>>
>>>> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase <https://github.com/cloudera-labs/SparkOnHBase>
>>>> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992> already integrates that feature into the hbase side, but
>>>> that feature has not been released.
>>>>
>>>> Best,
>>>> Sun.
>>>>
>>>> fightfate@163.com <ma...@163.com>
>>>>
>>>> From: censj <ma...@lotuseed.com>
>>>> Date: 2015-12-09 15:04
>>>> To: user@spark.apache.org <ma...@spark.apache.org>
>>>> Subject: About Spark On Hbase
>>>> hi all,
>>>> now I using spark,but I not found spark operation hbase open source. Do any one tell me?
>
Re: About Spark On Hbase
Posted by censj <ce...@lotuseed.com>.
Thank you! I know!!!!
> 在 2015年12月9日,15:59,fightfate@163.com 写道:
>
> If you are using maven , you can add the cloudera maven repo to the repository in pom.xml
> and add the dependency of spark-hbase.
> I just found this : http://spark-packages.org/package/nerdammer/spark-hbase-connector <http://spark-packages.org/package/nerdammer/spark-hbase-connector>
> as Feng Dongyu recommend, you can try this also, but I had no experience of using this.
>
>
> fightfate@163.com <ma...@163.com>
>
> 发件人: censj <ma...@lotuseed.com>
> 发送时间: 2015-12-09 15:44
> 收件人: fightfate@163.com <ma...@163.com>
> 抄送: user@spark.apache.org <ma...@spark.apache.org>
> 主题: Re: About Spark On Hbase
> So, I how to get this jar? I use set package project.I not found sbt lib.
>> 在 2015年12月9日,15:42,fightfate@163.com <ma...@163.com> 写道:
>>
>> I don't think it really need CDH component. Just use the API
>>
>> fightfate@163.com <ma...@163.com>
>>
>> 发件人: censj <ma...@lotuseed.com>
>> 发送时间: 2015-12-09 15:31
>> 收件人: fightfate@163.com <ma...@163.com>
>> 抄送: user@spark.apache.org <ma...@spark.apache.org>
>> 主题: Re: About Spark On Hbase
>> But this is dependent on CDH。I not install CDH。
>>> 在 2015年12月9日,15:18,fightfate@163.com <ma...@163.com> 写道:
>>>
>>> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase <https://github.com/cloudera-labs/SparkOnHBase>
>>> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992> already integrates that feature into the hbase side, but
>>> that feature has not been released.
>>>
>>> Best,
>>> Sun.
>>>
>>> fightfate@163.com <ma...@163.com>
>>>
>>> From: censj <ma...@lotuseed.com>
>>> Date: 2015-12-09 15:04
>>> To: user@spark.apache.org <ma...@spark.apache.org>
>>> Subject: About Spark On Hbase
>>> hi all,
>>> now I using spark,but I not found spark operation hbase open source. Do any one tell me?
回复: Re: About Spark On Hbase
Posted by "fightfate@163.com" <fi...@163.com>.
If you are using maven , you can add the cloudera maven repo to the repository in pom.xml
and add the dependency of spark-hbase.
I just found this : http://spark-packages.org/package/nerdammer/spark-hbase-connector
as Feng Dongyu recommend, you can try this also, but I had no experience of using this.
fightfate@163.com
发件人: censj
发送时间: 2015-12-09 15:44
收件人: fightfate@163.com
抄送: user@spark.apache.org
主题: Re: About Spark On Hbase
So, I how to get this jar? I use set package project.I not found sbt lib.
在 2015年12月9日,15:42,fightfate@163.com 写道:
I don't think it really need CDH component. Just use the API
fightfate@163.com
发件人: censj
发送时间: 2015-12-09 15:31
收件人: fightfate@163.com
抄送: user@spark.apache.org
主题: Re: About Spark On Hbase
But this is dependent on CDH。I not install CDH。
在 2015年12月9日,15:18,fightfate@163.com 写道:
Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase
Also, HBASE-13992 already integrates that feature into the hbase side, but
that feature has not been released.
Best,
Sun.
fightfate@163.com
From: censj
Date: 2015-12-09 15:04
To: user@spark.apache.org
Subject: About Spark On Hbase
hi all,
now I using spark,but I not found spark operation hbase open source. Do any one tell me?
Re: About Spark On Hbase
Posted by censj <ce...@lotuseed.com>.
So, I how to get this jar? I use set package project.I not found sbt lib.
> 在 2015年12月9日,15:42,fightfate@163.com 写道:
>
> I don't think it really need CDH component. Just use the API
>
> fightfate@163.com <ma...@163.com>
>
> 发件人: censj <ma...@lotuseed.com>
> 发送时间: 2015-12-09 15:31
> 收件人: fightfate@163.com <ma...@163.com>
> 抄送: user@spark.apache.org <ma...@spark.apache.org>
> 主题: Re: About Spark On Hbase
> But this is dependent on CDH。I not install CDH。
>> 在 2015年12月9日,15:18,fightfate@163.com <ma...@163.com> 写道:
>>
>> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase <https://github.com/cloudera-labs/SparkOnHBase>
>> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992> already integrates that feature into the hbase side, but
>> that feature has not been released.
>>
>> Best,
>> Sun.
>>
>> fightfate@163.com <ma...@163.com>
>>
>> From: censj <ma...@lotuseed.com>
>> Date: 2015-12-09 15:04
>> To: user@spark.apache.org <ma...@spark.apache.org>
>> Subject: About Spark On Hbase
>> hi all,
>> now I using spark,but I not found spark operation hbase open source. Do any one tell me?
回复: Re: About Spark On Hbase
Posted by "fightfate@163.com" <fi...@163.com>.
I don't think it really need CDH component. Just use the API
fightfate@163.com
发件人: censj
发送时间: 2015-12-09 15:31
收件人: fightfate@163.com
抄送: user@spark.apache.org
主题: Re: About Spark On Hbase
But this is dependent on CDH。I not install CDH。
在 2015年12月9日,15:18,fightfate@163.com 写道:
Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase
Also, HBASE-13992 already integrates that feature into the hbase side, but
that feature has not been released.
Best,
Sun.
fightfate@163.com
From: censj
Date: 2015-12-09 15:04
To: user@spark.apache.org
Subject: About Spark On Hbase
hi all,
now I using spark,but I not found spark operation hbase open source. Do any one tell me?
Re: About Spark On Hbase
Posted by censj <ce...@lotuseed.com>.
But this is dependent on CDH。I not install CDH。
> 在 2015年12月9日,15:18,fightfate@163.com 写道:
>
> Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase <https://github.com/cloudera-labs/SparkOnHBase>
> Also, HBASE-13992 <https://issues.apache.org/jira/browse/HBASE-13992> already integrates that feature into the hbase side, but
> that feature has not been released.
>
> Best,
> Sun.
>
> fightfate@163.com <ma...@163.com>
>
> From: censj <ma...@lotuseed.com>
> Date: 2015-12-09 15:04
> To: user@spark.apache.org <ma...@spark.apache.org>
> Subject: About Spark On Hbase
> hi all,
> now I using spark,but I not found spark operation hbase open source. Do any one tell me?
Re: About Spark On Hbase
Posted by "fightfate@163.com" <fi...@163.com>.
Actually you can refer to https://github.com/cloudera-labs/SparkOnHBase
Also, HBASE-13992 already integrates that feature into the hbase side, but
that feature has not been released.
Best,
Sun.
fightfate@163.com
From: censj
Date: 2015-12-09 15:04
To: user@spark.apache.org
Subject: About Spark On Hbase
hi all,
now I using spark,but I not found spark operation hbase open source. Do any one tell me?
Re: About Spark On Hbase
Posted by censj <ce...@lotuseed.com>.
Can you get me a example?
I want to update base data.
> 在 2015年12月9日,15:19,Fengdong Yu <fe...@everstring.com> 写道:
>
> https://github.com/nerdammer/spark-hbase-connector <https://github.com/nerdammer/spark-hbase-connector>
>
> This is better and easy to use.
>
>
>
>
>
>> On Dec 9, 2015, at 3:04 PM, censj <censj@lotuseed.com <ma...@lotuseed.com>> wrote:
>>
>> hi all,
>> now I using spark,but I not found spark operation hbase open source. Do any one tell me?
>>
>
Re: About Spark On Hbase
Posted by Fengdong Yu <fe...@everstring.com>.
https://github.com/nerdammer/spark-hbase-connector
This is better and easy to use.
> On Dec 9, 2015, at 3:04 PM, censj <ce...@lotuseed.com> wrote:
>
> hi all,
> now I using spark,but I not found spark operation hbase open source. Do any one tell me?
>