You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Nishant Aggarwal <ni...@gmail.com> on 2016/02/05 04:28:03 UTC

SQL Statement on DataFrame

Dear All,

I am working on a scenario mentioned below. Need your help:

Task:
Load the data from HBASE using Phoenix into Spark as a DataFrame, do the
operation and store the data back to HBASE using Phoenix. I know this is
feasible via writing code.

My question is, Is it possible to load the HBASE table using Phoenix into a
DataFrame and perform SQL queries on top of it(Instead of writing code) and
store the result back to HBASE ?

Any help on this will be highly appreciated.



Thanks and Regards
Nishant Aggarwal, PMP
Cell No:- +91 99588 94305
http://in.linkedin.com/pub/nishant-aggarwal/53/698/11b

Re: SQL Statement on DataFrame

Posted by Nishant Aggarwal <ni...@gmail.com>.

Hi Ted,

I am using Spark-Shell to do this.
I am using Phoenix's client jar for integrating Spark with HBASE. All the
operations will be done on Spark side.

Thanks,
Nishant


Thanks and Regards
Nishant Aggarwal, PMP
Cell No:- +91 99588 94305
http://in.linkedin.com/pub/nishant-aggarwal/53/698/11b


On Fri, Feb 5, 2016 at 9:04 AM, Ted Yu <yu...@gmail.com> wrote:

> Did you mean using bin/sqlline.py to perform the query ?
>
> Have you asked on Phoenix mailing list ?
>
> Phoenix has phoenix-spark module.
>
> Cheers
>
> On Thu, Feb 4, 2016 at 7:28 PM, Nishant Aggarwal <ni...@gmail.com>
> wrote:
>
>> Dear All,
>>
>> I am working on a scenario mentioned below. Need your help:
>>
>> Task:
>> Load the data from HBASE using Phoenix into Spark as a DataFrame, do the
>> operation and store the data back to HBASE using Phoenix. I know this is
>> feasible via writing code.
>>
>> My question is, Is it possible to load the HBASE table using Phoenix into
>> a DataFrame and perform SQL queries on top of it(Instead of writing code)
>> and store the result back to HBASE ?
>>
>> Any help on this will be highly appreciated.
>>
>>
>>
>> Thanks and Regards
>> Nishant Aggarwal, PMP
>> Cell No:- +91 99588 94305
>> http://in.linkedin.com/pub/nishant-aggarwal/53/698/11b
>>
>>
>

Re: SQL Statement on DataFrame

Posted by Ted Yu <yu...@gmail.com>.

Did you mean using bin/sqlline.py to perform the query ?

Have you asked on Phoenix mailing list ?

Phoenix has phoenix-spark module.

Cheers

On Thu, Feb 4, 2016 at 7:28 PM, Nishant Aggarwal <ni...@gmail.com>
wrote:

> Dear All,
>
> I am working on a scenario mentioned below. Need your help:
>
> Task:
> Load the data from HBASE using Phoenix into Spark as a DataFrame, do the
> operation and store the data back to HBASE using Phoenix. I know this is
> feasible via writing code.
>
> My question is, Is it possible to load the HBASE table using Phoenix into
> a DataFrame and perform SQL queries on top of it(Instead of writing code)
> and store the result back to HBASE ?
>
> Any help on this will be highly appreciated.
>
>
>
> Thanks and Regards
> Nishant Aggarwal, PMP
> Cell No:- +91 99588 94305
> http://in.linkedin.com/pub/nishant-aggarwal/53/698/11b
>
>