You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Hafiz Mujadid <ha...@gmail.com> on 2015/08/11 14:47:16 UTC
passing hbase scan start row in spark_phoenix
Hi all!
Can we use spark_phoenix in a way just like in normal java api we can pass
start row to filter data as follow
*//creating a scan object with start and stop row keys*
*Scan scan = new Scan(Bytes.ToBytes("a.b.x|1"),Bytes.toBytes("a.b.x|2"); *
Re: passing hbase scan start row in spark_phoenix
Posted by Hafiz Mujadid <ha...@gmail.com>.
thanks :)
On Wed, Aug 12, 2015 at 5:58 AM, Yuhao Bi <by...@gmail.com> wrote:
> Hi Hafiz,
>
> Sorry about the mistake, In 2), the returned records should be
> 6 ccc2 bbb ccc 2012 NULL
> 8 ccc4 bbb ccc 2014 NULL
>
> I hope I did not confuse you.
>
>
> Thanks.
>
> 2015-08-12 3:16 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
>
>> Hi Yuhao!
>>
>> Why record number 6 should be omitted?
>>
>> On Tue, Aug 11, 2015 at 9:08 PM, Yuhao Bi <by...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> 1.Image we create the test table with following sql:
>>> CREATE TABLE library (
>>> title varchar not null,
>>> author varchar not null,
>>> isbn varchar not null,
>>> published_date integer,
>>> description varchar,
>>> CONSTRAINT pk PRIMARY KEY(title, author, isbn)
>>> )
>>>
>>> 2.After insert some test data, our table may looks like,
>>>
>>> title author isbn published_date description
>>> 1 aaa1 aaa aaa 2010 NULL
>>> 2 aaa2 aaa aaa 2008 NULL
>>> 3 aaa3 aaa aaa 2007 NULL
>>> 4 bbb1 aaa bbb 2009 NULL
>>> 5 ccc1 aaa ccc 2011 NULL
>>> 6 ccc2 bbb ccc 2012 NULL
>>> 7 ccc3 bbb ccc 2006 NULL
>>> 8 ccc4 bbb ccc 2014 NULL
>>>
>>> 3.Then we use the query sql (from http://phoenix.apache.org/paged.html)
>>> SELECT title, author, isbn, description
>>> FROM library
>>> WHERE published_date > 2010
>>> AND (title, author, isbn) > (?, ?, ?) //this row value
>>> constructors will specify the start row
>>> ORDER BY title, author, isbn //In our case ORDER
>>> BY is ignored (CONSTRAINT pk PRIMARY KEY(title, author, isbn))
>>> LIMIT 20
>>>
>>> 1) SELECT title, author, isbn, description FROM library WHERE
>>> published_date > 2010 LIMIT 20;
>>> should return:
>>> title author isbn published_date description
>>> 1 aaa1 aaa aaa 2010 NULL
>>> 5 ccc1 aaa ccc 2011 NULL
>>> 6 ccc2 bbb ccc 2012 NULL
>>> 8 ccc4 bbb ccc 2014 NULL
>>> 2)SELECT title, author, isbn, description FROM library WHERE
>>> published_date > 2010
>>> AND (title, author, isbn) > ('ccc1', 'aaa', 'ccc') //this
>>> means we skip all rows in front of "record 5"(ccc2 bbb ccc 2012
>>> NULL)
>>>
>>> //and return matched records below "record 5" (exclude)
>>> LIMIT 20;
>>> should return:
>>> 7 ccc2 bbb ccc 2012 NULL
>>> 8 ccc4 bbb ccc 2014 NULL
>>>
>>>
>>> Sorry I dont't have a running cluster around, correct me if I was wrong.
>>> :)
>>>
>>>
>>> Thanks.
>>>
>>>
>>>
>>> 2015-08-11 22:43 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
>>>
>>>> Yes I want to do pagination and I am confused how to achieve pagination?
>>>>
>>>> On Tue, Aug 11, 2015 at 7:08 PM, Yuhao Bi <by...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Here is some official document which may help.
>>>>>
>>>>> 1. http://phoenix.apache.org/skip_scan.html
>>>>> 2. Wanna do some Pagination-like scan?
>>>>> Please refer to http://phoenix.apache.org/paged.html
>>>>>
>>>>> Thanks.
>>>>>
>>>>> 2015-08-11 20:47 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
>>>>>
>>>>>> Hi all!
>>>>>>
>>>>>>
>>>>>> Can we use spark_phoenix in a way just like in normal java api we can
>>>>>> pass start row to filter data as follow
>>>>>> *//creating a scan object with start and stop row keys*
>>>>>> *Scan scan = new
>>>>>> Scan(Bytes.ToBytes("a.b.x|1"),Bytes.toBytes("a.b.x|2"); *
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards: HAFIZ MUJADID
>>>>
>>>
>>>
>>
>>
>> --
>> Regards: HAFIZ MUJADID
>>
>
>
--
Regards: HAFIZ MUJADID
Re: passing hbase scan start row in spark_phoenix
Posted by Yuhao Bi <by...@gmail.com>.
Hi Hafiz,
Sorry about the mistake, In 2), the returned records should be
6 ccc2 bbb ccc 2012 NULL
8 ccc4 bbb ccc 2014 NULL
I hope I did not confuse you.
Thanks.
2015-08-12 3:16 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
> Hi Yuhao!
>
> Why record number 6 should be omitted?
>
> On Tue, Aug 11, 2015 at 9:08 PM, Yuhao Bi <by...@gmail.com> wrote:
>
>> Hi,
>>
>> 1.Image we create the test table with following sql:
>> CREATE TABLE library (
>> title varchar not null,
>> author varchar not null,
>> isbn varchar not null,
>> published_date integer,
>> description varchar,
>> CONSTRAINT pk PRIMARY KEY(title, author, isbn)
>> )
>>
>> 2.After insert some test data, our table may looks like,
>>
>> title author isbn published_date description
>> 1 aaa1 aaa aaa 2010 NULL
>> 2 aaa2 aaa aaa 2008 NULL
>> 3 aaa3 aaa aaa 2007 NULL
>> 4 bbb1 aaa bbb 2009 NULL
>> 5 ccc1 aaa ccc 2011 NULL
>> 6 ccc2 bbb ccc 2012 NULL
>> 7 ccc3 bbb ccc 2006 NULL
>> 8 ccc4 bbb ccc 2014 NULL
>>
>> 3.Then we use the query sql (from http://phoenix.apache.org/paged.html)
>> SELECT title, author, isbn, description
>> FROM library
>> WHERE published_date > 2010
>> AND (title, author, isbn) > (?, ?, ?) //this row value
>> constructors will specify the start row
>> ORDER BY title, author, isbn //In our case ORDER
>> BY is ignored (CONSTRAINT pk PRIMARY KEY(title, author, isbn))
>> LIMIT 20
>>
>> 1) SELECT title, author, isbn, description FROM library WHERE
>> published_date > 2010 LIMIT 20;
>> should return:
>> title author isbn published_date description
>> 1 aaa1 aaa aaa 2010 NULL
>> 5 ccc1 aaa ccc 2011 NULL
>> 6 ccc2 bbb ccc 2012 NULL
>> 8 ccc4 bbb ccc 2014 NULL
>> 2)SELECT title, author, isbn, description FROM library WHERE
>> published_date > 2010
>> AND (title, author, isbn) > ('ccc1', 'aaa', 'ccc') //this means
>> we skip all rows in front of "record 5"(ccc2 bbb ccc 2012
>> NULL)
>>
>> //and return matched records below "record 5" (exclude)
>> LIMIT 20;
>> should return:
>> 7 ccc2 bbb ccc 2012 NULL
>> 8 ccc4 bbb ccc 2014 NULL
>>
>>
>> Sorry I dont't have a running cluster around, correct me if I was wrong.
>> :)
>>
>>
>> Thanks.
>>
>>
>>
>> 2015-08-11 22:43 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
>>
>>> Yes I want to do pagination and I am confused how to achieve pagination?
>>>
>>> On Tue, Aug 11, 2015 at 7:08 PM, Yuhao Bi <by...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Here is some official document which may help.
>>>>
>>>> 1. http://phoenix.apache.org/skip_scan.html
>>>> 2. Wanna do some Pagination-like scan?
>>>> Please refer to http://phoenix.apache.org/paged.html
>>>>
>>>> Thanks.
>>>>
>>>> 2015-08-11 20:47 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
>>>>
>>>>> Hi all!
>>>>>
>>>>>
>>>>> Can we use spark_phoenix in a way just like in normal java api we can
>>>>> pass start row to filter data as follow
>>>>> *//creating a scan object with start and stop row keys*
>>>>> *Scan scan = new
>>>>> Scan(Bytes.ToBytes("a.b.x|1"),Bytes.toBytes("a.b.x|2"); *
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Regards: HAFIZ MUJADID
>>>
>>
>>
>
>
> --
> Regards: HAFIZ MUJADID
>
Re: passing hbase scan start row in spark_phoenix
Posted by Hafiz Mujadid <ha...@gmail.com>.
Hi Yuhao!
Why record number 6 should be omitted?
On Tue, Aug 11, 2015 at 9:08 PM, Yuhao Bi <by...@gmail.com> wrote:
> Hi,
>
> 1.Image we create the test table with following sql:
> CREATE TABLE library (
> title varchar not null,
> author varchar not null,
> isbn varchar not null,
> published_date integer,
> description varchar,
> CONSTRAINT pk PRIMARY KEY(title, author, isbn)
> )
>
> 2.After insert some test data, our table may looks like,
>
> title author isbn published_date description
> 1 aaa1 aaa aaa 2010 NULL
> 2 aaa2 aaa aaa 2008 NULL
> 3 aaa3 aaa aaa 2007 NULL
> 4 bbb1 aaa bbb 2009 NULL
> 5 ccc1 aaa ccc 2011 NULL
> 6 ccc2 bbb ccc 2012 NULL
> 7 ccc3 bbb ccc 2006 NULL
> 8 ccc4 bbb ccc 2014 NULL
>
> 3.Then we use the query sql (from http://phoenix.apache.org/paged.html)
> SELECT title, author, isbn, description
> FROM library
> WHERE published_date > 2010
> AND (title, author, isbn) > (?, ?, ?) //this row value
> constructors will specify the start row
> ORDER BY title, author, isbn //In our case ORDER
> BY is ignored (CONSTRAINT pk PRIMARY KEY(title, author, isbn))
> LIMIT 20
>
> 1) SELECT title, author, isbn, description FROM library WHERE
> published_date > 2010 LIMIT 20;
> should return:
> title author isbn published_date description
> 1 aaa1 aaa aaa 2010 NULL
> 5 ccc1 aaa ccc 2011 NULL
> 6 ccc2 bbb ccc 2012 NULL
> 8 ccc4 bbb ccc 2014 NULL
> 2)SELECT title, author, isbn, description FROM library WHERE
> published_date > 2010
> AND (title, author, isbn) > ('ccc1', 'aaa', 'ccc') //this means
> we skip all rows in front of "record 5"(ccc2 bbb ccc 2012
> NULL)
>
> //and return matched records below "record 5" (exclude)
> LIMIT 20;
> should return:
> 7 ccc2 bbb ccc 2012 NULL
> 8 ccc4 bbb ccc 2014 NULL
>
>
> Sorry I dont't have a running cluster around, correct me if I was wrong.
> :)
>
>
> Thanks.
>
>
>
> 2015-08-11 22:43 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
>
>> Yes I want to do pagination and I am confused how to achieve pagination?
>>
>> On Tue, Aug 11, 2015 at 7:08 PM, Yuhao Bi <by...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Here is some official document which may help.
>>>
>>> 1. http://phoenix.apache.org/skip_scan.html
>>> 2. Wanna do some Pagination-like scan?
>>> Please refer to http://phoenix.apache.org/paged.html
>>>
>>> Thanks.
>>>
>>> 2015-08-11 20:47 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
>>>
>>>> Hi all!
>>>>
>>>>
>>>> Can we use spark_phoenix in a way just like in normal java api we can
>>>> pass start row to filter data as follow
>>>> *//creating a scan object with start and stop row keys*
>>>> *Scan scan = new
>>>> Scan(Bytes.ToBytes("a.b.x|1"),Bytes.toBytes("a.b.x|2"); *
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Regards: HAFIZ MUJADID
>>
>
>
--
Regards: HAFIZ MUJADID
Re: passing hbase scan start row in spark_phoenix
Posted by Yuhao Bi <by...@gmail.com>.
Hi,
1.Image we create the test table with following sql:
CREATE TABLE library (
title varchar not null,
author varchar not null,
isbn varchar not null,
published_date integer,
description varchar,
CONSTRAINT pk PRIMARY KEY(title, author, isbn)
)
2.After insert some test data, our table may looks like,
title author isbn published_date description
1 aaa1 aaa aaa 2010 NULL
2 aaa2 aaa aaa 2008 NULL
3 aaa3 aaa aaa 2007 NULL
4 bbb1 aaa bbb 2009 NULL
5 ccc1 aaa ccc 2011 NULL
6 ccc2 bbb ccc 2012 NULL
7 ccc3 bbb ccc 2006 NULL
8 ccc4 bbb ccc 2014 NULL
3.Then we use the query sql (from http://phoenix.apache.org/paged.html)
SELECT title, author, isbn, description
FROM library
WHERE published_date > 2010
AND (title, author, isbn) > (?, ?, ?) //this row value
constructors will specify the start row
ORDER BY title, author, isbn //In our case ORDER BY
is ignored (CONSTRAINT pk PRIMARY KEY(title, author, isbn))
LIMIT 20
1) SELECT title, author, isbn, description FROM library WHERE
published_date > 2010 LIMIT 20;
should return:
title author isbn published_date description
1 aaa1 aaa aaa 2010 NULL
5 ccc1 aaa ccc 2011 NULL
6 ccc2 bbb ccc 2012 NULL
8 ccc4 bbb ccc 2014 NULL
2)SELECT title, author, isbn, description FROM library WHERE
published_date > 2010
AND (title, author, isbn) > ('ccc1', 'aaa', 'ccc') //this means
we skip all rows in front of "record 5"(ccc2 bbb ccc 2012
NULL)
//and return matched records below "record 5" (exclude)
LIMIT 20;
should return:
7 ccc2 bbb ccc 2012 NULL
8 ccc4 bbb ccc 2014 NULL
Sorry I dont't have a running cluster around, correct me if I was wrong.
:)
Thanks.
2015-08-11 22:43 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
> Yes I want to do pagination and I am confused how to achieve pagination?
>
> On Tue, Aug 11, 2015 at 7:08 PM, Yuhao Bi <by...@gmail.com> wrote:
>
>> Hi,
>>
>> Here is some official document which may help.
>>
>> 1. http://phoenix.apache.org/skip_scan.html
>> 2. Wanna do some Pagination-like scan?
>> Please refer to http://phoenix.apache.org/paged.html
>>
>> Thanks.
>>
>> 2015-08-11 20:47 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
>>
>>> Hi all!
>>>
>>>
>>> Can we use spark_phoenix in a way just like in normal java api we can
>>> pass start row to filter data as follow
>>> *//creating a scan object with start and stop row keys*
>>> *Scan scan = new
>>> Scan(Bytes.ToBytes("a.b.x|1"),Bytes.toBytes("a.b.x|2"); *
>>>
>>>
>>>
>>
>
>
> --
> Regards: HAFIZ MUJADID
>
Re: passing hbase scan start row in spark_phoenix
Posted by Hafiz Mujadid <ha...@gmail.com>.
Yes I want to do pagination and I am confused how to achieve pagination?
On Tue, Aug 11, 2015 at 7:08 PM, Yuhao Bi <by...@gmail.com> wrote:
> Hi,
>
> Here is some official document which may help.
>
> 1. http://phoenix.apache.org/skip_scan.html
> 2. Wanna do some Pagination-like scan?
> Please refer to http://phoenix.apache.org/paged.html
>
> Thanks.
>
> 2015-08-11 20:47 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
>
>> Hi all!
>>
>>
>> Can we use spark_phoenix in a way just like in normal java api we can
>> pass start row to filter data as follow
>> *//creating a scan object with start and stop row keys*
>> *Scan scan = new Scan(Bytes.ToBytes("a.b.x|1"),Bytes.toBytes("a.b.x|2"); *
>>
>>
>>
>
--
Regards: HAFIZ MUJADID
Re: passing hbase scan start row in spark_phoenix
Posted by Yuhao Bi <by...@gmail.com>.
Hi,
Here is some official document which may help.
1. http://phoenix.apache.org/skip_scan.html
2. Wanna do some Pagination-like scan?
Please refer to http://phoenix.apache.org/paged.html
Thanks.
2015-08-11 20:47 GMT+08:00 Hafiz Mujadid <ha...@gmail.com>:
> Hi all!
>
>
> Can we use spark_phoenix in a way just like in normal java api we can pass
> start row to filter data as follow
> *//creating a scan object with start and stop row keys*
> *Scan scan = new Scan(Bytes.ToBytes("a.b.x|1"),Bytes.toBytes("a.b.x|2"); *
>
>
>