You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@phoenix.apache.org by Li Gao <ga...@marinsoftware.com> on 2015/12/15 23:57:13 UTC

Does phoenix spark support arbitrary SELECT statement?

Hi community,

Does Phoenix Spark support arbitrary SELECT statements for generating DF or
RDD?

>From this reading: https://phoenix.apache.org/phoenix_spark.html I am not
sure how to do that.

Thanks,
Li

Re: Does phoenix spark support arbitrary SELECT statement?

Posted by Li Gao <ga...@marinsoftware.com>.

Thanks Josh, I will take a look at the link.

- Li

On Tue, Dec 15, 2015 at 3:07 PM, Josh Mahonin <jm...@gmail.com> wrote:

> Hi Li,
>
> When using the DataFrame integration, it supports arbitrary SELECT
> statements. Column pruning and predicate filtering is pushed down to
> Phoenix, and aggregate functions are executed within Spark.
>
> When using RDDs directly, you can specify a table name, columns and an
> optional WHERE predicate for basic filtering. Aggregate functions however
> are not supported.
>
> The integration tests have a reasonably thorough set of examples on both
> DataFrames and RDDs with Phoenix. [1]
>
> Good luck,
>
> Josh
>
> [1]
> https://github.com/apache/phoenix/blob/master/phoenix-spark/src/it/scala/org/apache/phoenix/spark/PhoenixSparkIT.scala
>
> On Tue, Dec 15, 2015 at 5:57 PM, Li Gao <ga...@marinsoftware.com> wrote:
>
>> Hi community,
>>
>> Does Phoenix Spark support arbitrary SELECT statements for generating DF
>> or RDD?
>>
>> From this reading: https://phoenix.apache.org/phoenix_spark.html I am
>> not sure how to do that.
>>
>> Thanks,
>> Li
>>
>>
>

Re: Does phoenix spark support arbitrary SELECT statement?

Posted by Josh Mahonin <jm...@gmail.com>.

Hi Li,

When using the DataFrame integration, it supports arbitrary SELECT
statements. Column pruning and predicate filtering is pushed down to
Phoenix, and aggregate functions are executed within Spark.

When using RDDs directly, you can specify a table name, columns and an
optional WHERE predicate for basic filtering. Aggregate functions however
are not supported.

The integration tests have a reasonably thorough set of examples on both
DataFrames and RDDs with Phoenix. [1]

Good luck,

Josh

[1]
https://github.com/apache/phoenix/blob/master/phoenix-spark/src/it/scala/org/apache/phoenix/spark/PhoenixSparkIT.scala

On Tue, Dec 15, 2015 at 5:57 PM, Li Gao <ga...@marinsoftware.com> wrote:

> Hi community,
>
> Does Phoenix Spark support arbitrary SELECT statements for generating DF
> or RDD?
>
> From this reading: https://phoenix.apache.org/phoenix_spark.html I am not
> sure how to do that.
>
> Thanks,
> Li
>
>